The important part before starting to do any thing, we understand the problem very well to be able to be soundness about features you need and features have effect on your prediction.
One of the biggest problems when people prepare to post a new listing on airbnb is, how much should one ask for? Of course the most intuitive way is to check how other similar postings price their apartment/house.
Our goal of this problem is to predict the listing price based on the listing characteristics 🔥🔥, in this way to optimize user experience and lower the bar to be a new host😍 !
This is a multimodality task which is to estimate the price of an Airbnb listing using multi class classification based on the photo and description provided.
The price of a new listing, as well as the type of rental being promoted, is predicted using a
!pip install langdetect
!pip install deep_translator
!pip install -q focal-loss
!pip install seaborn
Requirement already satisfied: langdetect in c:\users\lab\.conda\envs\mlpr\lib\site-packages (1.0.9) Requirement already satisfied: six in c:\users\lab\.conda\envs\mlpr\lib\site-packages (from langdetect) (1.16.0) Requirement already satisfied: deep_translator in c:\users\lab\.conda\envs\mlpr\lib\site-packages (1.10.1) Requirement already satisfied: requests<3.0.0,>=2.23.0 in c:\users\lab\.conda\envs\mlpr\lib\site-packages (from deep_translator) (2.28.1) Requirement already satisfied: beautifulsoup4<5.0.0,>=4.9.1 in c:\users\lab\.conda\envs\mlpr\lib\site-packages (from deep_translator) (4.11.1) Requirement already satisfied: soupsieve>1.2 in c:\users\lab\.conda\envs\mlpr\lib\site-packages (from beautifulsoup4<5.0.0,>=4.9.1->deep_translator) (2.3.2.post1) Requirement already satisfied: urllib3<1.27,>=1.21.1 in c:\users\lab\.conda\envs\mlpr\lib\site-packages (from requests<3.0.0,>=2.23.0->deep_translator) (1.26.14) Requirement already satisfied: certifi>=2017.4.17 in c:\users\lab\.conda\envs\mlpr\lib\site-packages (from requests<3.0.0,>=2.23.0->deep_translator) (2022.12.7) Requirement already satisfied: charset-normalizer<3,>=2 in c:\users\lab\.conda\envs\mlpr\lib\site-packages (from requests<3.0.0,>=2.23.0->deep_translator) (2.0.4) Requirement already satisfied: idna<4,>=2.5 in c:\users\lab\.conda\envs\mlpr\lib\site-packages (from requests<3.0.0,>=2.23.0->deep_translator) (3.4) Requirement already satisfied: seaborn in c:\users\lab\.conda\envs\mlpr\lib\site-packages (0.12.2) Requirement already satisfied: matplotlib!=3.6.1,>=3.1 in c:\users\lab\.conda\envs\mlpr\lib\site-packages (from seaborn) (3.6.2) Requirement already satisfied: numpy!=1.24.0,>=1.17 in c:\users\lab\.conda\envs\mlpr\lib\site-packages (from seaborn) (1.23.5) Requirement already satisfied: pandas>=0.25 in c:\users\lab\.conda\envs\mlpr\lib\site-packages (from seaborn) (1.5.2) Requirement already satisfied: cycler>=0.10 in c:\users\lab\.conda\envs\mlpr\lib\site-packages (from matplotlib!=3.6.1,>=3.1->seaborn) (0.11.0) Requirement already satisfied: pyparsing>=2.2.1 in c:\users\lab\.conda\envs\mlpr\lib\site-packages (from matplotlib!=3.6.1,>=3.1->seaborn) (3.0.9) Requirement already satisfied: pillow>=6.2.0 in c:\users\lab\.conda\envs\mlpr\lib\site-packages (from matplotlib!=3.6.1,>=3.1->seaborn) (9.3.0) Requirement already satisfied: python-dateutil>=2.7 in c:\users\lab\.conda\envs\mlpr\lib\site-packages (from matplotlib!=3.6.1,>=3.1->seaborn) (2.8.2) Requirement already satisfied: fonttools>=4.22.0 in c:\users\lab\.conda\envs\mlpr\lib\site-packages (from matplotlib!=3.6.1,>=3.1->seaborn) (4.25.0) Requirement already satisfied: kiwisolver>=1.0.1 in c:\users\lab\.conda\envs\mlpr\lib\site-packages (from matplotlib!=3.6.1,>=3.1->seaborn) (1.4.4) Requirement already satisfied: contourpy>=1.0.1 in c:\users\lab\.conda\envs\mlpr\lib\site-packages (from matplotlib!=3.6.1,>=3.1->seaborn) (1.0.5) Requirement already satisfied: packaging>=20.0 in c:\users\lab\.conda\envs\mlpr\lib\site-packages (from matplotlib!=3.6.1,>=3.1->seaborn) (22.0) Requirement already satisfied: pytz>=2020.1 in c:\users\lab\.conda\envs\mlpr\lib\site-packages (from pandas>=0.25->seaborn) (2022.7) Requirement already satisfied: six>=1.5 in c:\users\lab\.conda\envs\mlpr\lib\site-packages (from python-dateutil>=2.7->matplotlib!=3.6.1,>=3.1->seaborn) (1.16.0)
import numpy as np # linear algebra
import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)
import seaborn as sns
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
%matplotlib inline
from PIL import Image
from langdetect import detect
from deep_translator import GoogleTranslator
from focal_loss import SparseCategoricalFocalLoss
# preprocess text data
from tensorflow.keras.preprocessing.text import Tokenizer
from tensorflow.keras.preprocessing.sequence import pad_sequences
from pprint import pprint
import collections
import matplotlib.pyplot as plt
import numpy as np
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.layers import Conv2D, Flatten, Dense, MaxPool2D , Dropout
from tensorflow.keras.optimizers import Adam
from tensorflow.data import AUTOTUNE
from tensorflow.keras.models import save_model
data = pd.read_csv(r"C:\Users\LAB\Videos\flash_new_1\copy-of-cisc-873-dm-w23-a4\a4\train_xy.csv")
submission_data = pd.read_csv(r"C:\Users\LAB\Videos\flash_new_1\copy-of-cisc-873-dm-w23-a4\a4\test_x.csv")
print('Shape of data' ,data.shape)
print('Shape of submission_data ', submission_data.shape)
Shape of data (7627, 4) Shape of submission_data (7360, 3)
data.head()
| summary | image | type | price | |
|---|---|---|---|---|
| 0 | Spacious, sunny and cozy modern apartment in t... | img_train/0.jpg | Apartment | 1 |
| 1 | Located in one of the most vibrant and accessi... | img_train/1.jpg | Apartment | 0 |
| 2 | Logement coquet et douillet à 10 minutes du ce... | img_train/2.jpg | Apartment | 1 |
| 3 | Beautiful and spacious (1076 sc ft, / 100 mc) ... | img_train/3.jpg | Apartment | 1 |
| 4 | Très grand appartement ''rustique'' et très ag... | img_train/4.jpg | Apartment | 0 |
submission_data.head()
| id | summary | image | |
|---|---|---|---|
| 0 | 0 | Charming warm house is ready to host you here ... | img_test/0.jpg |
| 1 | 1 | La chambre est spacieuse et lumineuse, dans un... | img_test/1.jpg |
| 2 | 2 | Grande chambre confortable située au sous-sol ... | img_test/2.jpg |
| 3 | 3 | Près d’un Métro, ligne orange. 10 minutes à pi... | img_test/3.jpg |
| 4 | 4 | Very bright appartment and very cosy. 2 separa... | img_test/4.jpg |
data['image'] = 'C:\\Users\\LAB\\Videos\\flash_new_1\\copy-of-cisc-873-dm-w23-a4\\a4\\'+data['image']
submission_data['image'] ='C:\\Users\\LAB\\Videos\\flash_new_1\\copy-of-cisc-873-dm-w23-a4\\a4\\'+submission_data['image']
data.head()
| summary | image | type | price | translated_summary | |
|---|---|---|---|---|---|
| 0 | Spacious, sunny and cozy modern apartment in t... | C:\Users\LAB\Videos\flash_new_1\copy-of-cisc-8... | Apartment | 1 | Spacious, sunny and cozy modern apartment in t... |
| 1 | Located in one of the most vibrant and accessi... | C:\Users\LAB\Videos\flash_new_1\copy-of-cisc-8... | Apartment | 0 | Located in one of the most vibrant and accessi... |
| 2 | Logement coquet et douillet à 10 minutes du ce... | C:\Users\LAB\Videos\flash_new_1\copy-of-cisc-8... | Apartment | 1 | Pretty and cozy accommodation 10 minutes from ... |
| 3 | Beautiful and spacious (1076 sc ft, / 100 mc) ... | C:\Users\LAB\Videos\flash_new_1\copy-of-cisc-8... | Apartment | 1 | Beautiful and spacious (1076 sc ft, / 100 mc) ... |
| 4 | Très grand appartement ''rustique'' et très ag... | C:\Users\LAB\Videos\flash_new_1\copy-of-cisc-8... | Apartment | 0 | Very large ''rustic'' and very pleasant apartm... |
Obeservation
Let's move on to explore and know more about our data
class Explore_Data:
def __init__(self, dataframe):
self.dataframe = dataframe
#------------------------------- Check null ^_^ ------------------------------------
'''
This function counts null in each column in the dataframe and calculate the percent of nulls in the column then return the
dataframe consist of 2 columns : one contains count of null values in each column and second contains percent
'''
def null_values(self , plot =True):
null_val = pd.DataFrame(self.dataframe.isnull().sum())
null_val.columns = ['null_val']
null_val['percent_'] = round(null_val['null_val'] / len(self.dataframe.index), 2) * 100
null_val = null_val.sort_values('percent_', ascending = False)
if plot:
sns.heatmap(self.dataframe.isnull(), cbar=False)
return null_val
#------------------------------- Check duplication ^_^ ------------------------------------
'''
This function counts duplicated rows in the dataframe
'''
def duplicated_values(self):
return print("Number of duplicated rows" , self.dataframe.duplicated().sum())
#------------------------------- CHECK CONSTANT FEATURES ^_^ ------------------------------------
'''
This function returns the columns that contain one value a cross all samples
'''
def constant_columns(self):
constant_columns = [[col ,self.dataframe[col].value_counts()] for col in self.dataframe.columns if (self.dataframe[col].nunique()) == 1]
return constant_columns
#--------------------------------------- cardinality ^_^ ---------------------------------------
'''
calculate unique values in each column and returns dataframe consists of count and percent. This helps us to find column that have
high cardinality
'''
def cardinality(self):
unique_val = pd.DataFrame(np.array([len(self.dataframe[col].unique()) for col in self.dataframe.columns ]) , index=self.dataframe.columns)
unique_val.columns = ['unique_val']
unique_val['percent_'] = round(unique_val['unique_val'] / len(self.dataframe.index), 2) * 100
unique_val = unique_val.sort_values('percent_', ascending = False)
return unique_val
#--------------------------------------- drop columns ^_^ ---------------------------------------
'''
This function drop rows that contain null values.
'''
def drop_col(self):
self.dataframe = self.dataframe.dropna(axis=0)
return self.dataframe
#--------------------------------------- count of each category ^_^ ---------------------------------------
'''
This function returns the count of each category in the column as percetage. Also, it put them in a nice plot to imgine the difference between them.
'''
def imbalance(self,col):
print(self.dataframe[col].value_counts(normalize=True))
self.dataframe[col].value_counts().plot(kind='bar')
#--------------------------------------- display images ^_^ ---------------------------------------
'''
This function helps us to display sample of images. This functio catakes image as an image or as an array. If you will pass an image, put image argument with true and img argument with None.
If you will pass an array, put image argument with false and img argument with array values.
'''
def display_images(self,col,image=True , img =None):
ROW_IMG = 5
N_ROWS = 5
plt.figure(figsize=(25,20))
for index in range(1, ROW_IMG * N_ROWS + 1):
plt.subplot(N_ROWS, ROW_IMG, index)
plt.axis('off')
if image:
img = mpimg.imread(self.dataframe[col][index])
imgplot = plt.imshow(img)
else:
imgplot = plt.imshow(img[index,:,:,0])
plt.title('image with index [{}]'.format(index))
print("Shape of image " , img.shape)
#--------------------------------------- drop columns ^_^ ---------------------------------------
'''
This function helps us to count words in each row and returns how many words the text samples tend to have. Also print the statistic summary of count_worsd
'''
def count_word(self):
self.dataframe['summary']= self.dataframe['summary'].astype('category')
count_word = pd.DataFrame(self.dataframe.summary.map(lambda summary: len(summary.split(" "))))
count_word.columns = ['count_word']
print("Describe count of words ")
print("_"*50)
print(self.dataframe.summary.map(lambda summary: len(summary.split(" "))).describe())
return count_word
Explore = Explore_Data(data)
data.info()
<class 'pandas.core.frame.DataFrame'> RangeIndex: 7627 entries, 0 to 7626 Data columns (total 4 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 summary 7326 non-null object 1 image 7627 non-null object 2 type 7627 non-null object 3 price 7627 non-null int64 dtypes: int64(1), object(3) memory usage: 238.5+ KB
Observation
data.describe(include=['O'])
| summary | image | type | |
|---|---|---|---|
| count | 7326 | 7627 | 7627 |
| unique | 6844 | 7627 | 24 |
| top | Featured in The New York Times, The Wall Stree... | img_train/0.jpg | Apartment |
| freq | 88 | 1 | 5765 |
Observation
Explore.display_images('image')
Shape of image (480, 1010, 3) Shape of image (480, 640, 3) Shape of image (480, 720, 3) Shape of image (480, 640, 3) Shape of image (480, 360, 3) Shape of image (480, 720, 3) Shape of image (480, 640, 3) Shape of image (480, 721, 3) Shape of image (480, 752, 3) Shape of image (480, 720, 3) Shape of image (480, 640, 3) Shape of image (480, 720, 3) Shape of image (480, 720, 3) Shape of image (480, 720, 3) Shape of image (480, 360, 3) Shape of image (480, 698, 3) Shape of image (480, 720, 3) Shape of image (480, 720, 3) Shape of image (480, 640, 3) Shape of image (480, 721, 3) Shape of image (480, 720, 3) Shape of image (480, 640, 3) Shape of image (480, 723, 3) Shape of image (480, 643, 3) Shape of image (480, 720, 3)
# plot image and its summary
plt.figure(figsize=(25,20))
img = mpimg.imread(data['image'][0])
imgplot = plt.imshow(img)
plt.title(data['summary'][0])
Text(0.5, 1.0, 'Spacious, sunny and cozy modern apartment in the heart of Montreal. This 3 bedroom centrally located in the very popular Plateau Mont-royal neighborhood in the middle of prince-arthur pedestrian-only street. Close to all amenities ( Restaurants, coffee house, bars, clubs, shopping, universities, subway stations). Experience Montreal like a real local resident, be in the heart of the action: grand prix week, grill Saint-Laurent festival, mural 2019 and so much more!')
plt.figure(figsize=(25,20))
img = mpimg.imread(data['image'][1])
imgplot = plt.imshow(img)
print(img.shape)
plt.title(data['summary'][1])
(480, 1010, 3)
Text(0.5, 1.0, 'Located in one of the most vibrant and accessible locations of downtown Montreal, this one bedroom condo will not only impress you but leave you with one of the most memorable experiences. It is walking distance of the popular Sainte-Catherine Street, the Bell Center, the Old Port, Lachine Canal, Bonaventure metro, and much much more...')
Observation
Explore.count_word()
Describe count of words __________________________________________________ count 7326.000000 mean 58.002457 std 23.267883 min 1.000000 25% 40.000000 50% 61.000000 75% 78.000000 max 177.000000 Name: summary, dtype: float64
| count_word | |
|---|---|
| 0 | 70.0 |
| 1 | 55.0 |
| 2 | 28.0 |
| 3 | 94.0 |
| 4 | 88.0 |
| ... | ... |
| 7622 | 57.0 |
| 7623 | 41.0 |
| 7624 | 53.0 |
| 7625 | 31.0 |
| 7626 | 28.0 |
7627 rows × 1 columns
Observations
data['summary'].head()
0 Spacious, sunny and cozy modern apartment in t... 1 Located in one of the most vibrant and accessi... 2 Logement coquet et douillet à 10 minutes du ce... 3 Beautiful and spacious (1076 sc ft, / 100 mc) ... 4 Très grand appartement ''rustique'' et très ag... Name: summary, dtype: object
data['summary'].tail()
7622 Un grand logement 4 et 1/2, tout inclut, bien ... 7623 Magnificent condo directly on the river. You w... 7624 This apartment is perfect for anyone visiting ... 7625 It is a cozy ,clean ,and comfortable apartment... 7626 Modern country style (newly-renovated); open c... Name: summary, dtype: object
Observation
Explore.null_values()
| null_val | percent_ | |
|---|---|---|
| summary | 301 | 4.0 |
| image | 0 | 0.0 |
| type | 0 | 0.0 |
| price | 0 | 0.0 |
Observations
Explore.duplicated_values()
Number of duplicated rows 0
Observation
Explore.cardinality()
| unique_val | percent_ | |
|---|---|---|
| image | 7627 | 100.0 |
| summary | 6845 | 90.0 |
| type | 24 | 0.0 |
| price | 3 | 0.0 |
Observation
Explore.imbalance('price')
0 0.621083 1 0.315065 2 0.063852 Name: price, dtype: float64
Observation
Explore.imbalance('type')
Apartment 0.755867 Condominium 0.090599 House 0.053232 Loft 0.042481 Townhouse 0.021896 Serviced apartment 0.010096 Bed and breakfast 0.004982 Guest suite 0.004196 Hostel 0.003409 Bungalow 0.003278 Guesthouse 0.001836 Cottage 0.001573 Aparthotel 0.001573 Boutique hotel 0.001311 Other 0.001049 Villa 0.000918 Tiny house 0.000393 Boat 0.000262 Cabin 0.000262 Camper/RV 0.000262 Casa particular (Cuba) 0.000131 Hotel 0.000131 Earth house 0.000131 Castle 0.000131 Name: type, dtype: float64
Observations
data = Explore.drop_col()
print("Shape of data after droping rows that contain null vales " ,data.shape)
(7326, 4)
Observation
#simple function to detect and translate text
def detect_and_translate(text):
try:
result_lang = detect(text)
if result_lang == 'en':
return text
else:
translator = GoogleTranslator(source='auto', target='en')
translate_text = translator.translate(text)
return translate_text
except:
return text
def _preprocess(list_of_text):
return pad_sequences(tokenizer.texts_to_sequences(list_of_text),maxlen=max_len,padding='post',)
data.summary = data.summary.astype('str')
data['translated_summary'] = data['summary'].apply( detect_and_translate)
submission_data['translated_summary'] = submission_data['summary'].apply( detect_and_translate)
data = data.to_csv('data_translated.csv', index=False)
submission_data = submission_data.to_csv('submission_data_translated.csv', index=False)
vocab_size = 40000
max_len = 177
# build vocabulary from training set
tokenizer = Tokenizer(num_words=vocab_size)
tokenizer.fit_on_texts(data['translated_summary'])
# # padding is done inside:
x_train_text_id = _preprocess(data['translated_summary'])
print( tokenizer.sequences_to_texts(x_train_text_id[:1]))
print(x_train_text_id[0])
['spacious sunny and cozy modern apartment in the heart of montreal this 3 bedroom centrally located in the very popular plateau mont royal neighborhood in the middle of prince arthur pedestrian only street close to all amenities restaurants coffee house bars clubs shopping universities subway stations experience montreal like a real local resident be in the heart of the action grand prix week grill saint laurent festival mural 2019 and so much more']
[ 95 224 2 91 119 10 5 1 28 6 13 23 78 34
602 15 5 1 27 556 48 50 35 90 5 1 585 6
1040 1041 824 143 26 25 4 24 154 21 137 134 53 406
140 614 126 218 235 13 212 3 703 279 2550 72 5 1
28 6 1 530 969 1236 892 2116 156 188 603 1845 1479 2
191 410 158 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0]
# In 'LA' mode, 'L' stands for Luminance (8-bit pixels, black and white) and 'A' stands for Alpha. The value of alpha in 'LA' mode can range from 0 to 255.
# So when you convert an image to 'LA' mode using convert(), you’re converting it to black and white with an alpha channel.
# In digital images, each pixel contains color information (such as values describing intensity of red, green, and blue) and also contains a value for its opacity known as its ‘alpha’ value1. An alpha value of 1 means totally opaque, and an alpha value of 0 means totally transparent1.
# Opacity refers to the degree to which something is transparent or opaque. In digital images, opacity is represented by an alpha channel. An alpha value of 1 means totally opaque, and an alpha value of 0 means totally transparent.
def Resize_image(filepath):
try:
# Convert to 'LA' mode and resize image to 128 x 128
image = Image.open(filepath).convert('LA').resize((128,128))
arr = np.array(image)
except:
print(filepath)
arr = np.zeros((128, 128, 2))
return arr
#resize trained images
x_train_image = np.array([Resize_image(i) for i in data.image])
C:\Users\LAB\Videos\flash_new_1\copy-of-cisc-873-dm-w23-a4\a4\img_train/4176.jpg C:\Users\LAB\Videos\flash_new_1\copy-of-cisc-873-dm-w23-a4\a4\img_train/6083.jpg
# display the images after resizing
Explore_Data(data).display_images('image_arr',image=False , img = x_train_image )
Shape of image (7326, 128, 128, 2) Shape of image (7326, 128, 128, 2) Shape of image (7326, 128, 128, 2) Shape of image (7326, 128, 128, 2) Shape of image (7326, 128, 128, 2) Shape of image (7326, 128, 128, 2) Shape of image (7326, 128, 128, 2) Shape of image (7326, 128, 128, 2) Shape of image (7326, 128, 128, 2) Shape of image (7326, 128, 128, 2) Shape of image (7326, 128, 128, 2) Shape of image (7326, 128, 128, 2) Shape of image (7326, 128, 128, 2) Shape of image (7326, 128, 128, 2) Shape of image (7326, 128, 128, 2) Shape of image (7326, 128, 128, 2) Shape of image (7326, 128, 128, 2) Shape of image (7326, 128, 128, 2) Shape of image (7326, 128, 128, 2) Shape of image (7326, 128, 128, 2) Shape of image (7326, 128, 128, 2) Shape of image (7326, 128, 128, 2) Shape of image (7326, 128, 128, 2) Shape of image (7326, 128, 128, 2) Shape of image (7326, 128, 128, 2)
data.tail()
| summary | image | type | price | translated_summary | |
|---|---|---|---|---|---|
| 7321 | Un grand logement 4 et 1/2, tout inclut, bien ... | C:\Users\LAB\Videos\flash_new_1\copy-of-cisc-8... | Apartment | 0 | A large 4 and 1/2 apartment, all inclusive, we... |
| 7322 | Magnificent condo directly on the river. You w... | C:\Users\LAB\Videos\flash_new_1\copy-of-cisc-8... | Apartment | 2 | Magnificent condo directly on the river. You w... |
| 7323 | This apartment is perfect for anyone visiting ... | C:\Users\LAB\Videos\flash_new_1\copy-of-cisc-8... | Apartment | 1 | This apartment is perfect for anyone visiting ... |
| 7324 | It is a cozy ,clean ,and comfortable apartment... | C:\Users\LAB\Videos\flash_new_1\copy-of-cisc-8... | Apartment | 0 | It is a cozy ,clean ,and comfortable apartment... |
| 7325 | Modern country style (newly-renovated); open c... | C:\Users\LAB\Videos\flash_new_1\copy-of-cisc-8... | House | 1 | Modern country style (newly-renovated); open c... |
# labels:
data['type'] = data.type.astype('category').cat.codes
len_type = len(data.type.unique())
len_price = len(data.price.unique())
print("Number of categories in type column " , len_type )
print("Number of categories in price column" , len_price)
Number of categories in type column 24 Number of categories in price column 3
Explore.imbalance('type')
1 0.755867 10 0.090599 17 0.053232 18 0.042481 22 0.021896 20 0.010096 2 0.004982 13 0.004196 15 0.003409 5 0.003278 14 0.001836 11 0.001573 0 0.001573 4 0.001311 19 0.001049 23 0.000918 21 0.000393 3 0.000262 6 0.000262 7 0.000262 8 0.000131 16 0.000131 12 0.000131 9 0.000131 Name: type, dtype: float64
Observation
x_test_image = np.array([Resize_image(i) for i in submission_data['image']])
submission_data.summary = submission_data.summary.astype('str')
x_test_text = _preprocess(submission_data['translated_summary'].astype('str'))# load and resize the images from the test set
# here we have two inputs. one for image and the other for text.
x_train_text_id = np.array(x_train_text_id, dtype = 'float32')
in_text = keras.Input(batch_shape=(None, max_len))
in_image = keras.Input(batch_shape=(None,128,128, 2))
checkpoint_filepath = '/tmp/checkpoint'
#--------------------------------------- Multimodality model ^_^ ----------------------------------------------
'''
This Function helps us build multi modality model through combining text and image features. Then pass them to output dense layer with softmax activation function
After building model, we complie it with optimizer, loss values for each task, and loss_weights for each task. This function gives us
options our loss function and loss_weights for each class. It's a very powerful way to try more trails without repeat code
we try Focal Loss which is, by design, a better choice for a loss function when dealing with imbalanced classification datasets.
'''
def build_model(averaged , flattened , price_weight , type_weight , weights_focal = False):
# fusion - combinig both
fused = tf.concat([averaged, flattened], axis=-1)
# multi-task learning (each is a multi-class classification)
# one dense layer for each task
p_price = Dense(len_price, activation='softmax', name='price')(fused)
p_type = Dense(len_type, activation='softmax', name='type')(fused)
# define model input/output using keys.
model = keras.Model(
inputs={
'summary': in_text,
'image': in_image
},
outputs={
'price': p_price,
'type': p_type,
},
)
if weights_focal :
model.compile(
optimizer=Adam(),
loss={
'price': SparseCategoricalFocalLoss(gamma=2),
'type': SparseCategoricalFocalLoss(gamma=2),
},
loss_weights={
'price': price_weight,
'type': type_weight,
},
metrics={
'price': ['SparseCategoricalAccuracy'],
'type': ['SparseCategoricalAccuracy'],
},
)
else:
model.compile(
optimizer=Adam(),
loss={
'price': 'sparse_categorical_crossentropy',
'type': 'sparse_categorical_crossentropy',
},
loss_weights={
'price': price_weight,
'type': type_weight,
},
metrics={
'price': ['SparseCategoricalAccuracy'],
'type': ['SparseCategoricalAccuracy'],
},
)
model.summary()
return model
#--------------------------------------- model one Input ^_^ ----------------------------------------------
'''
This Function is same as above one but this time we pass only one feature from 2 features (text and image).We built this function t
to see the performance in each case and compare it with the multi modality model
'''
def build_model_one_input(model_structure , input_feature , price_weight , type_weight , weights_focal = False):
p_price = Dense(len_price, activation='softmax', name='price')(model_structure)
p_type = Dense(len_type, activation='softmax', name='type')(model_structure)
# define model input/output using keys.
model = keras.Model(
inputs=input_feature ,
outputs={
'price': p_price,
'type': p_type,
},
)
if weights_focal :
model.compile(
optimizer=Adam(),
loss={
'price': SparseCategoricalFocalLoss(gamma=2),
'type': SparseCategoricalFocalLoss(gamma=2),
},
loss_weights={
'price': price_weight,
'type': type_weight,
},
metrics={
'price': ['SparseCategoricalAccuracy'],
'type': ['SparseCategoricalAccuracy'],
},
)
else:
model.compile(
optimizer=Adam(),
loss={
'price': 'sparse_categorical_crossentropy',
'type': 'sparse_categorical_crossentropy',
},
loss_weights={
'price': price_weight,
'type': type_weight,
},
metrics={
'price': ['SparseCategoricalAccuracy'],
'type': ['SparseCategoricalAccuracy'],
},
)
model.summary()
return model
#--------------------------------------- Image Part ^_^ ----------------------------------------------
'''
This Function build convolution layers to extract features from images.
Through this Function, we can specify dropout layers and convolution layers.
if you want to add more convolution layers, pass true to an argument conv2d.
if you want to add dropout layer followed each convolution layer, pass true to an argument drop.
if you want to add dropout and convolution layer, pass true to both arguments
Then we pass the return of this function to build_model or build_model_one_input Function
'''
def image_model(conv2d =False ,drop = False):
cov = Conv2D(64, (5, 5), activation='relu')(in_image)
if drop:
cov = Dropout(.2)(cov)
pl = MaxPool2D((2, 2))(cov)
if conv2d :
cov1 = Conv2D(64, (7, 7), activation='relu')(pl)
if drop:
cov1 = Dropout(.4)(cov1)
pl = MaxPool2D((2, 2))(cov1)
cov2 = Conv2D(32, (16, 16), activation='relu')(pl)
if drop:
cov2 = Dropout(.4)(cov2)
pl2 = MaxPool2D((2, 2))(cov2)
flattened = Flatten()(pl2)
return flattened
#--------------------------------------- Text Part ^_^ ----------------------------------------------
'''
This Function can build different type of models such lstm or gru or bidirectional lstm or just
applying embedding in the text to extract features from the text.
Through this Function, we can specify type of model we need.
if you want to apply lstm, pass a word of 'lstm'.
if you want to apply gru, pass a word of 'gru'.
if you want to apply bidirectional lstm, pass a word of 'bidirect'.
if you want to apply embbeding only, pass any word other than the previous words.
'''
def text_model(type_model):
embedded = keras.layers.Embedding(tokenizer.num_words, 177)(in_text)
if type_model == 'lstm':
text_model_ = tf.keras.layers.LSTM(16)(embedded)
elif type_model == 'gru':
text_model_ =tf.keras.layers.GRU(7)(embedded)
elif type_model == 'bidirect':
text_model_ = tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(5))(embedded)
else:
# simple average of embedding
text_model_ = tf.reduce_mean(embedded, axis=1)
return text_model_
#--------------------------------------- Train Multi Modality Model ^_^ ----------------------------------------------
'''
This Function takes the model we built already from the previous fuction and train it. we specify some of hyperparamters
through try and error way to tune them to improve the performance
'''
def fit_model(model ):
model_checkpoint_callback = tf.keras.callbacks.ModelCheckpoint(filepath=checkpoint_filepath, monitor='val_accuracy',mode='max',save_best_only=True)
history = model.fit(
x={
'summary': x_train_text_id,
'image': x_train_image
},
y={
'price': tf.convert_to_tensor(data.price),
'type': tf.convert_to_tensor(data.type),
},
epochs=50,
batch_size=200,
validation_split=0.2,
callbacks=[tf.keras.callbacks.EarlyStopping(monitor='val_price_loss', patience=5 ) , model_checkpoint_callback],
workers = -1 ,
use_multiprocessing =True,
verbose=1)
return history ,model
#--------------------------------------- Train mono Model ^_^ ----------------------------------------------
'''
This Function has the same functionality of the above function but the only difference that it deals with model with single input
'''
def fit_model_one_input(model , input_features):
model_checkpoint_callback = tf.keras.callbacks.ModelCheckpoint(filepath=checkpoint_filepath, monitor='val_accuracy',mode='max',save_best_only=True)
history = model.fit(
x=input_features,
y={
'price': tf.convert_to_tensor(data.price),
'type': tf.convert_to_tensor(data.type),
},
epochs=50,
batch_size=200,
validation_split=0.2,
callbacks=[tf.keras.callbacks.EarlyStopping(monitor='val_price_loss', patience=5 ) , model_checkpoint_callback],
workers = -1 ,
use_multiprocessing =True,
verbose=1)
return history ,model
#--------------------------------------- Visulaization ^_^ ----------------------------------------------
'''
This Function plot a nice 2 graphs : one for loss and the other for accuracy.
'''
def plot_loss_accuracy_curves(history):
# first we display the training price accuracy vs. the validation price accuracy
plt.plot(history.history['price_sparse_categorical_accuracy'])
plt.plot(history.history['val_price_sparse_categorical_accuracy'])
plt.legend(['accuracy', 'val_accuracy'], loc='upper left')
plt.show() # display the figure
print("____"*50)
# first we display the training price loss vs. the validation price loss
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.legend(['loss', 'val_loss'], loc='upper left')
plt.show() # display the figure
#--------------------------------------- Save Csv submitted file of multimodality model ^_^ ----------------------------------------------
'''
This Function takes the trained model and applies the multi modality model on the test data. these prediction is saved into csv file to submit it
'''
def Submission(model , model_name):
# we can do prediction on training set
y_predict = model.predict(
{
'summary': x_test_text,
'image': x_test_image
}
)
# probabilities
price_predicted = y_predict['price']
print("probabilities of predicted price" )
print( price_predicted)# probability the sample belongs to each of the categories
print("__"*50)
# categories
price_category_predicted = np.argmax(price_predicted, axis=1) # pick the highest value, and assign it as the predicted class
print("values of of predicted price :" ) # print the categorical predictions, which are in [0,1,2]
print(price_category_predicted)
# create the submission dataframe/csv for submissionpd.
submitted_file = pd.DataFrame({'id': submission_data.id,'price': price_category_predicted})
print("counts of classes predicted in submitted file")
print(submitted_file.value_counts)
submitted_file.to_csv('submission_{}.csv'.format(model_name), index=False)
#--------------------------------------- Save Csv submitted file of mono model ^_^ ----------------------------------------------
'''
This Function has the same functionality of the above function but the only difference that it deals with model with single input
'''
def submission_one_input(model , model_name , feature_input):
y_predict = model.predict(feature_input)
# probabilities
price_predicted = y_predict['price']
print("probabilities of predicted price" )
print( price_predicted)# probability the sample belongs to each of the categories
print("__"*50)
# categories
price_category_predicted = np.argmax(price_predicted, axis=1) # pick the highest value, and assign it as the predicted class
print("values of of predicted price :" ) # print the categorical predictions, which are in [0,1,2]
print(price_category_predicted)
# create the submission dataframe/csv for submissionpd.
submitted_file = pd.DataFrame({'id': submission_data.id,'price': price_category_predicted})
print("counts of classes predicted in submitted file")
print(submitted_file.value_counts)
submitted_file.to_csv('submission_{}.csv'.format(model_name), index=False)
#------------------------- Save Model ^_^ ------------------------------------
def model_save(model , model_name):
save_model(model, "model{}.h5".format(model_name))
In this trail, I prefered to starting with building multimodality model. As we mentioned before, we expect that 2 features together( text , image) will give us high performance.
In this trail, we handled:
Text part:
We used Embedding layer enables us to convert each word into a fixed length vector of defined size. The resultant vector is a dense one with having real values instead of just 0's and 1's. The fixed length of word vectors helps us to represent words in a better way along with reduced dimensions.
Then we used averaged layer to compute the mean of the embeddings to reduce the rank of the embedding.
Image part:
we used convolutional neural network layer with 64 filters, and a 5x5 mask, no padding , one stride with relu as activation function .
Then we used max pooling layer with a 2x2 mask,it reduced the size to 62x62x64
Followed by another convolutional neural network layer with 32 filters, and a 16x16 mask, no padding , one stride .
Then we used max pooling layer with a 2x2 mask,it reduced the size to 23x23x32
After that, we used flatten layer to flatten the multi-dimensional input tensors into a single dimension that it was (16928,).
Imbalance in price column :
averaged_1st_trail = text_model('embedding')
flattened_1st_trail = image_model()
model_1st_trail = build_model(averaged_1st_trail , flattened_1st_trail , 1 , 0 , weights_focal = True)
history_model_1st_trail , model_1st_trail = fit_model(model_1st_trail)
Model: "model_1"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_2 (InputLayer) [(None, 128, 128, 2 0 []
)]
conv2d_2 (Conv2D) (None, 124, 124, 64 3264 ['input_2[0][0]']
)
max_pooling2d_2 (MaxPooling2D) (None, 62, 62, 64) 0 ['conv2d_2[0][0]']
input_1 (InputLayer) [(None, 177)] 0 []
conv2d_3 (Conv2D) (None, 47, 47, 32) 524320 ['max_pooling2d_2[0][0]']
embedding_1 (Embedding) (None, 177, 177) 7080000 ['input_1[0][0]']
max_pooling2d_3 (MaxPooling2D) (None, 23, 23, 32) 0 ['conv2d_3[0][0]']
tf.math.reduce_mean_1 (TFOpLam (None, 177) 0 ['embedding_1[0][0]']
bda)
flatten_1 (Flatten) (None, 16928) 0 ['max_pooling2d_3[0][0]']
tf.concat_1 (TFOpLambda) (None, 17105) 0 ['tf.math.reduce_mean_1[0][0]',
'flatten_1[0][0]']
price (Dense) (None, 3) 51318 ['tf.concat_1[0][0]']
type (Dense) (None, 24) 410544 ['tf.concat_1[0][0]']
==================================================================================================
Total params: 8,069,446
Trainable params: 8,069,446
Non-trainable params: 0
__________________________________________________________________________________________________
Epoch 1/50
30/30 [==============================] - ETA: 0s - loss: 10.8422 - price_loss: 10.8422 - type_loss: 16.0716 - price_sparse_categorical_accuracy: 0.3242 - type_sparse_categorical_accuracy: 0.0019 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 349s 12s/step - loss: 10.8422 - price_loss: 10.8422 - type_loss: 16.0716 - price_sparse_categorical_accuracy: 0.3242 - type_sparse_categorical_accuracy: 0.0019 - val_loss: 11.0496 - val_price_loss: 11.0496 - val_type_loss: 16.1181 - val_price_sparse_categorical_accuracy: 0.3145 - val_type_sparse_categorical_accuracy: 0.0000e+00
Epoch 2/50
30/30 [==============================] - ETA: 0s - loss: 6.4857 - price_loss: 6.4857 - type_loss: 15.7896 - price_sparse_categorical_accuracy: 0.5976 - type_sparse_categorical_accuracy: 0.0188 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 355s 12s/step - loss: 6.4857 - price_loss: 6.4857 - type_loss: 15.7896 - price_sparse_categorical_accuracy: 0.5976 - type_sparse_categorical_accuracy: 0.0188 - val_loss: 6.0580 - val_price_loss: 6.0580 - val_type_loss: 14.6726 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0819
Epoch 3/50
30/30 [==============================] - ETA: 0s - loss: 6.1309 - price_loss: 6.1309 - type_loss: 13.8898 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.1287 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 358s 12s/step - loss: 6.1309 - price_loss: 6.1309 - type_loss: 13.8898 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.1287 - val_loss: 6.0580 - val_price_loss: 6.0580 - val_type_loss: 13.5462 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.1501
Epoch 4/50
30/30 [==============================] - ETA: 0s - loss: 6.1309 - price_loss: 6.1309 - type_loss: 13.3984 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.1587 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 346s 12s/step - loss: 6.1309 - price_loss: 6.1309 - type_loss: 13.3984 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.1587 - val_loss: 6.0580 - val_price_loss: 6.0580 - val_type_loss: 13.4839 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.1542
Epoch 5/50
30/30 [==============================] - ETA: 0s - loss: 6.1309 - price_loss: 6.1309 - type_loss: 13.3711 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.1601 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 357s 12s/step - loss: 6.1309 - price_loss: 6.1309 - type_loss: 13.3711 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.1601 - val_loss: 6.0580 - val_price_loss: 6.0580 - val_type_loss: 13.4811 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.1555
Epoch 6/50
30/30 [==============================] - ETA: 0s - loss: 6.1309 - price_loss: 6.1309 - type_loss: 13.3698 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.1601 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 355s 12s/step - loss: 6.1309 - price_loss: 6.1309 - type_loss: 13.3698 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.1601 - val_loss: 6.0580 - val_price_loss: 6.0580 - val_type_loss: 13.4809 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.1555
Epoch 7/50
30/30 [==============================] - ETA: 0s - loss: 6.1309 - price_loss: 6.1309 - type_loss: 13.3698 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.1601 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 361s 12s/step - loss: 6.1309 - price_loss: 6.1309 - type_loss: 13.3698 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.1601 - val_loss: 6.0580 - val_price_loss: 6.0580 - val_type_loss: 13.4809 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.1555
plot_loss_accuracy_curves(history_model_1st_trail)
________________________________________________________________________________________________________________________________________________________________________________________________________
Observation
Submission(model_1st_trail , "1st_trail")
model_save(model_1st_trail , "1st_trail")
230/230 [==============================] - 63s 273ms/step probabilities of predicted price [[0.99999994 0. 0. ] [0.99999994 0. 0. ] [0.99999994 0. 0. ] ... [0.99999994 0. 0. ] [0.99999994 0. 0. ] [0.99999994 0. 0. ]] ____________________________________________________________________________________________________ values of of predicted price : [0 0 0 ... 0 0 0] counts of classes predicted in submitted file <bound method DataFrame.value_counts of id price 0 0 0 1 1 0 2 2 0 3 3 0 4 4 0 ... ... ... 7355 7626 0 7356 7627 0 7357 7628 0 7358 7629 0 7359 7630 0 [7360 rows x 2 columns]>
Observation
Let's try to improve thsis
#clear session
tf.keras.backend.clear_session()
In this trail, we will use the same model of the previous trail but this time, we will change the loss function to see if this will improve the performance.
Acutually, I expect that there will be an overfitting in the model. but I want to see the behaviour of this model in kaggle.
In this trail, we handled:
Text part:
We used Embedding layer enables us to convert each word into a fixed length vector of defined size. The resultant vector is a dense one with having real values instead of just 0's and 1's. The fixed length of word vectors helps us to represent words in a better way along with reduced dimensions.
Then we used averaged layer to compute the mean of the embeddings to reduce the rank of the embedding.
Image part:
we used convolutional neural network layer with 64 filters, and a 5x5 mask, no padding , one stride with relu as activation function .
Then we used max pooling layer with a 2x2 mask,it reduced the size to 62x62x64
Followed by another convolutional neural network layer with 32 filters, and a 16x16 mask, no padding , one stride .
Then we used max pooling layer with a 2x2 mask,it reduced the size to 23x23x32
After that, we used flatten layer to flatten the multi-dimensional input tensors into a single dimension that it was (16928,).
averaged_2nd_trail = text_model('embedding')
flattened_2nd_trail = image_model()
model_2dn_trail = build_model(averaged_2nd_trail , flattened_2nd_trail , 1 , 0 , weights_focal = False)
history_model_2nd_trail , model_2nd_trail = fit_model(model_2dn_trail)
Model: "model"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_2 (InputLayer) [(None, 128, 128, 2 0 []
)]
conv2d (Conv2D) (None, 124, 124, 64 3264 ['input_2[0][0]']
)
max_pooling2d (MaxPooling2D) (None, 62, 62, 64) 0 ['conv2d[0][0]']
input_1 (InputLayer) [(None, 177)] 0 []
conv2d_1 (Conv2D) (None, 47, 47, 32) 524320 ['max_pooling2d[0][0]']
embedding (Embedding) (None, 177, 177) 7080000 ['input_1[0][0]']
max_pooling2d_1 (MaxPooling2D) (None, 23, 23, 32) 0 ['conv2d_1[0][0]']
tf.math.reduce_mean (TFOpLambd (None, 177) 0 ['embedding[0][0]']
a)
flatten (Flatten) (None, 16928) 0 ['max_pooling2d_1[0][0]']
tf.concat (TFOpLambda) (None, 17105) 0 ['tf.math.reduce_mean[0][0]',
'flatten[0][0]']
price (Dense) (None, 3) 51318 ['tf.concat[0][0]']
type (Dense) (None, 24) 410544 ['tf.concat[0][0]']
==================================================================================================
Total params: 8,069,446
Trainable params: 8,069,446
Non-trainable params: 0
__________________________________________________________________________________________________
Epoch 1/50
30/30 [==============================] - ETA: 0s - loss: 171.5359 - price_loss: 171.5359 - type_loss: 107.0730 - price_sparse_categorical_accuracy: 0.4580 - type_sparse_categorical_accuracy: 0.0099 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 351s 12s/step - loss: 171.5359 - price_loss: 171.5359 - type_loss: 107.0730 - price_sparse_categorical_accuracy: 0.4580 - type_sparse_categorical_accuracy: 0.0099 - val_loss: 1.0879 - val_price_loss: 1.0879 - val_type_loss: 3.1778 - val_price_sparse_categorical_accuracy: 0.3199 - val_type_sparse_categorical_accuracy: 6.8213e-04
Epoch 2/50
30/30 [==============================] - ETA: 0s - loss: 1.0484 - price_loss: 1.0484 - type_loss: 3.1771 - price_sparse_categorical_accuracy: 0.6070 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 356s 12s/step - loss: 1.0484 - price_loss: 1.0484 - type_loss: 3.1771 - price_sparse_categorical_accuracy: 0.6070 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.9754 - val_price_loss: 0.9754 - val_type_loss: 3.1761 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 6.8213e-04
Epoch 3/50
30/30 [==============================] - ETA: 0s - loss: 0.9030 - price_loss: 0.9030 - type_loss: 3.1755 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 343s 11s/step - loss: 0.9030 - price_loss: 0.9030 - type_loss: 3.1755 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.8406 - val_price_loss: 0.8406 - val_type_loss: 3.1745 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 6.8213e-04
Epoch 4/50
30/30 [==============================] - ETA: 0s - loss: 0.8357 - price_loss: 0.8357 - type_loss: 3.1746 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 340s 11s/step - loss: 0.8357 - price_loss: 0.8357 - type_loss: 3.1746 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.8219 - val_price_loss: 0.8219 - val_type_loss: 3.1744 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 6.8213e-04
Epoch 5/50
30/30 [==============================] - ETA: 0s - loss: 0.8280 - price_loss: 0.8280 - type_loss: 3.1746 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 340s 11s/step - loss: 0.8280 - price_loss: 0.8280 - type_loss: 3.1746 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.8185 - val_price_loss: 0.8185 - val_type_loss: 3.1744 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 6.8213e-04
Epoch 6/50
30/30 [==============================] - ETA: 0s - loss: 0.8246 - price_loss: 0.8246 - type_loss: 3.1747 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 338s 11s/step - loss: 0.8246 - price_loss: 0.8246 - type_loss: 3.1747 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.8157 - val_price_loss: 0.8157 - val_type_loss: 3.1746 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 6.8213e-04
Epoch 7/50
30/30 [==============================] - ETA: 0s - loss: 0.8207 - price_loss: 0.8207 - type_loss: 3.1750 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 341s 11s/step - loss: 0.8207 - price_loss: 0.8207 - type_loss: 3.1750 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.8123 - val_price_loss: 0.8123 - val_type_loss: 3.1748 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 6.8213e-04
Epoch 8/50
30/30 [==============================] - ETA: 0s - loss: 0.8162 - price_loss: 0.8162 - type_loss: 3.1752 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 340s 11s/step - loss: 0.8162 - price_loss: 0.8162 - type_loss: 3.1752 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.8083 - val_price_loss: 0.8083 - val_type_loss: 3.1752 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 6.8213e-04
Epoch 9/50
30/30 [==============================] - ETA: 0s - loss: 0.8105 - price_loss: 0.8105 - type_loss: 3.1757 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 345s 12s/step - loss: 0.8105 - price_loss: 0.8105 - type_loss: 3.1757 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.8030 - val_price_loss: 0.8030 - val_type_loss: 3.1756 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 6.8213e-04
Epoch 10/50
30/30 [==============================] - ETA: 0s - loss: 0.8031 - price_loss: 0.8031 - type_loss: 3.1761 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 339s 11s/step - loss: 0.8031 - price_loss: 0.8031 - type_loss: 3.1761 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.7964 - val_price_loss: 0.7964 - val_type_loss: 3.1761 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 6.8213e-04
Epoch 11/50
30/30 [==============================] - ETA: 0s - loss: 0.7938 - price_loss: 0.7938 - type_loss: 3.1764 - price_sparse_categorical_accuracy: 0.6224 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 344s 11s/step - loss: 0.7938 - price_loss: 0.7938 - type_loss: 3.1764 - price_sparse_categorical_accuracy: 0.6224 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.7890 - val_price_loss: 0.7890 - val_type_loss: 3.1767 - val_price_sparse_categorical_accuracy: 0.6310 - val_type_sparse_categorical_accuracy: 6.8213e-04
Epoch 12/50
30/30 [==============================] - ETA: 0s - loss: 0.7829 - price_loss: 0.7829 - type_loss: 3.1770 - price_sparse_categorical_accuracy: 0.6261 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 339s 11s/step - loss: 0.7829 - price_loss: 0.7829 - type_loss: 3.1770 - price_sparse_categorical_accuracy: 0.6261 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.7786 - val_price_loss: 0.7786 - val_type_loss: 3.1770 - val_price_sparse_categorical_accuracy: 0.6330 - val_type_sparse_categorical_accuracy: 6.8213e-04
Epoch 13/50
30/30 [==============================] - ETA: 0s - loss: 0.7691 - price_loss: 0.7691 - type_loss: 3.1773 - price_sparse_categorical_accuracy: 0.6295 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 339s 11s/step - loss: 0.7691 - price_loss: 0.7691 - type_loss: 3.1773 - price_sparse_categorical_accuracy: 0.6295 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.7682 - val_price_loss: 0.7682 - val_type_loss: 3.1773 - val_price_sparse_categorical_accuracy: 0.6357 - val_type_sparse_categorical_accuracy: 6.8213e-04
Epoch 14/50
30/30 [==============================] - ETA: 0s - loss: 0.7553 - price_loss: 0.7553 - type_loss: 3.1776 - price_sparse_categorical_accuracy: 0.6379 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 339s 11s/step - loss: 0.7553 - price_loss: 0.7553 - type_loss: 3.1776 - price_sparse_categorical_accuracy: 0.6379 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.7576 - val_price_loss: 0.7576 - val_type_loss: 3.1775 - val_price_sparse_categorical_accuracy: 0.6392 - val_type_sparse_categorical_accuracy: 6.8213e-04
Epoch 15/50
30/30 [==============================] - ETA: 0s - loss: 0.7405 - price_loss: 0.7405 - type_loss: 3.1778 - price_sparse_categorical_accuracy: 0.6497 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 338s 11s/step - loss: 0.7405 - price_loss: 0.7405 - type_loss: 3.1778 - price_sparse_categorical_accuracy: 0.6497 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.7482 - val_price_loss: 0.7482 - val_type_loss: 3.1779 - val_price_sparse_categorical_accuracy: 0.6569 - val_type_sparse_categorical_accuracy: 6.8213e-04
Epoch 16/50
30/30 [==============================] - ETA: 0s - loss: 0.7267 - price_loss: 0.7267 - type_loss: 3.1780 - price_sparse_categorical_accuracy: 0.6589 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 338s 11s/step - loss: 0.7267 - price_loss: 0.7267 - type_loss: 3.1780 - price_sparse_categorical_accuracy: 0.6589 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.7391 - val_price_loss: 0.7391 - val_type_loss: 3.1780 - val_price_sparse_categorical_accuracy: 0.6583 - val_type_sparse_categorical_accuracy: 6.8213e-04
Epoch 17/50
30/30 [==============================] - ETA: 0s - loss: 0.7129 - price_loss: 0.7129 - type_loss: 3.1782 - price_sparse_categorical_accuracy: 0.6730 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 337s 11s/step - loss: 0.7129 - price_loss: 0.7129 - type_loss: 3.1782 - price_sparse_categorical_accuracy: 0.6730 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.7306 - val_price_loss: 0.7306 - val_type_loss: 3.1780 - val_price_sparse_categorical_accuracy: 0.6583 - val_type_sparse_categorical_accuracy: 6.8213e-04
Epoch 18/50
30/30 [==============================] - ETA: 0s - loss: 0.7005 - price_loss: 0.7005 - type_loss: 3.1782 - price_sparse_categorical_accuracy: 0.6799 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 338s 11s/step - loss: 0.7005 - price_loss: 0.7005 - type_loss: 3.1782 - price_sparse_categorical_accuracy: 0.6799 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.7235 - val_price_loss: 0.7235 - val_type_loss: 3.1781 - val_price_sparse_categorical_accuracy: 0.6678 - val_type_sparse_categorical_accuracy: 6.8213e-04
Epoch 19/50
30/30 [==============================] - ETA: 0s - loss: 0.6891 - price_loss: 0.6891 - type_loss: 3.1782 - price_sparse_categorical_accuracy: 0.6877 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 339s 11s/step - loss: 0.6891 - price_loss: 0.6891 - type_loss: 3.1782 - price_sparse_categorical_accuracy: 0.6877 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.7176 - val_price_loss: 0.7176 - val_type_loss: 3.1782 - val_price_sparse_categorical_accuracy: 0.6767 - val_type_sparse_categorical_accuracy: 6.8213e-04
Epoch 20/50
30/30 [==============================] - ETA: 0s - loss: 0.6772 - price_loss: 0.6772 - type_loss: 3.1783 - price_sparse_categorical_accuracy: 0.6991 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 338s 11s/step - loss: 0.6772 - price_loss: 0.6772 - type_loss: 3.1783 - price_sparse_categorical_accuracy: 0.6991 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.7125 - val_price_loss: 0.7125 - val_type_loss: 3.1781 - val_price_sparse_categorical_accuracy: 0.6774 - val_type_sparse_categorical_accuracy: 6.8213e-04
Epoch 21/50
30/30 [==============================] - ETA: 0s - loss: 0.6667 - price_loss: 0.6667 - type_loss: 3.1782 - price_sparse_categorical_accuracy: 0.7019 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 340s 11s/step - loss: 0.6667 - price_loss: 0.6667 - type_loss: 3.1782 - price_sparse_categorical_accuracy: 0.7019 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.7088 - val_price_loss: 0.7088 - val_type_loss: 3.1782 - val_price_sparse_categorical_accuracy: 0.6903 - val_type_sparse_categorical_accuracy: 6.8213e-04
Epoch 22/50
30/30 [==============================] - ETA: 0s - loss: 0.6563 - price_loss: 0.6563 - type_loss: 3.1783 - price_sparse_categorical_accuracy: 0.7128 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 339s 11s/step - loss: 0.6563 - price_loss: 0.6563 - type_loss: 3.1783 - price_sparse_categorical_accuracy: 0.7128 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.7039 - val_price_loss: 0.7039 - val_type_loss: 3.1781 - val_price_sparse_categorical_accuracy: 0.6828 - val_type_sparse_categorical_accuracy: 6.8213e-04
Epoch 23/50
30/30 [==============================] - ETA: 0s - loss: 0.6468 - price_loss: 0.6468 - type_loss: 3.1783 - price_sparse_categorical_accuracy: 0.7189 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 339s 11s/step - loss: 0.6468 - price_loss: 0.6468 - type_loss: 3.1783 - price_sparse_categorical_accuracy: 0.7189 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.7011 - val_price_loss: 0.7011 - val_type_loss: 3.1781 - val_price_sparse_categorical_accuracy: 0.6842 - val_type_sparse_categorical_accuracy: 6.8213e-04
Epoch 24/50
30/30 [==============================] - ETA: 0s - loss: 0.6370 - price_loss: 0.6370 - type_loss: 3.1783 - price_sparse_categorical_accuracy: 0.7220 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 339s 11s/step - loss: 0.6370 - price_loss: 0.6370 - type_loss: 3.1783 - price_sparse_categorical_accuracy: 0.7220 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.6970 - val_price_loss: 0.6970 - val_type_loss: 3.1781 - val_price_sparse_categorical_accuracy: 0.6924 - val_type_sparse_categorical_accuracy: 6.8213e-04
Epoch 25/50
30/30 [==============================] - ETA: 0s - loss: 0.6279 - price_loss: 0.6279 - type_loss: 3.1782 - price_sparse_categorical_accuracy: 0.7285 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 338s 11s/step - loss: 0.6279 - price_loss: 0.6279 - type_loss: 3.1782 - price_sparse_categorical_accuracy: 0.7285 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.6946 - val_price_loss: 0.6946 - val_type_loss: 3.1780 - val_price_sparse_categorical_accuracy: 0.6944 - val_type_sparse_categorical_accuracy: 6.8213e-04
Epoch 26/50
30/30 [==============================] - ETA: 0s - loss: 0.6202 - price_loss: 0.6202 - type_loss: 3.1782 - price_sparse_categorical_accuracy: 0.7316 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 338s 11s/step - loss: 0.6202 - price_loss: 0.6202 - type_loss: 3.1782 - price_sparse_categorical_accuracy: 0.7316 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.6921 - val_price_loss: 0.6921 - val_type_loss: 3.1780 - val_price_sparse_categorical_accuracy: 0.6992 - val_type_sparse_categorical_accuracy: 6.8213e-04
Epoch 27/50
30/30 [==============================] - ETA: 0s - loss: 0.6103 - price_loss: 0.6103 - type_loss: 3.1780 - price_sparse_categorical_accuracy: 0.7415 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 338s 11s/step - loss: 0.6103 - price_loss: 0.6103 - type_loss: 3.1780 - price_sparse_categorical_accuracy: 0.7415 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.6915 - val_price_loss: 0.6915 - val_type_loss: 3.1779 - val_price_sparse_categorical_accuracy: 0.6992 - val_type_sparse_categorical_accuracy: 6.8213e-04
Epoch 28/50
30/30 [==============================] - ETA: 0s - loss: 0.6016 - price_loss: 0.6016 - type_loss: 3.1780 - price_sparse_categorical_accuracy: 0.7452 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 338s 11s/step - loss: 0.6016 - price_loss: 0.6016 - type_loss: 3.1780 - price_sparse_categorical_accuracy: 0.7452 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.6893 - val_price_loss: 0.6893 - val_type_loss: 3.1779 - val_price_sparse_categorical_accuracy: 0.7019 - val_type_sparse_categorical_accuracy: 6.8213e-04
Epoch 29/50
30/30 [==============================] - ETA: 0s - loss: 0.5929 - price_loss: 0.5929 - type_loss: 3.1779 - price_sparse_categorical_accuracy: 0.7500 - type_sparse_categorical_accuracy: 3.4130e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 338s 11s/step - loss: 0.5929 - price_loss: 0.5929 - type_loss: 3.1779 - price_sparse_categorical_accuracy: 0.7500 - type_sparse_categorical_accuracy: 3.4130e-04 - val_loss: 0.6874 - val_price_loss: 0.6874 - val_type_loss: 3.1777 - val_price_sparse_categorical_accuracy: 0.7019 - val_type_sparse_categorical_accuracy: 6.8213e-04
Epoch 30/50
30/30 [==============================] - ETA: 0s - loss: 0.5855 - price_loss: 0.5855 - type_loss: 3.1778 - price_sparse_categorical_accuracy: 0.7529 - type_sparse_categorical_accuracy: 3.4130e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 340s 11s/step - loss: 0.5855 - price_loss: 0.5855 - type_loss: 3.1778 - price_sparse_categorical_accuracy: 0.7529 - type_sparse_categorical_accuracy: 3.4130e-04 - val_loss: 0.6865 - val_price_loss: 0.6865 - val_type_loss: 3.1778 - val_price_sparse_categorical_accuracy: 0.7026 - val_type_sparse_categorical_accuracy: 6.8213e-04
Epoch 31/50
30/30 [==============================] - ETA: 0s - loss: 0.5764 - price_loss: 0.5764 - type_loss: 3.1778 - price_sparse_categorical_accuracy: 0.7585 - type_sparse_categorical_accuracy: 6.8259e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 341s 11s/step - loss: 0.5764 - price_loss: 0.5764 - type_loss: 3.1778 - price_sparse_categorical_accuracy: 0.7585 - type_sparse_categorical_accuracy: 6.8259e-04 - val_loss: 0.6853 - val_price_loss: 0.6853 - val_type_loss: 3.1776 - val_price_sparse_categorical_accuracy: 0.7046 - val_type_sparse_categorical_accuracy: 6.8213e-04
Epoch 32/50
30/30 [==============================] - ETA: 0s - loss: 0.5689 - price_loss: 0.5689 - type_loss: 3.1778 - price_sparse_categorical_accuracy: 0.7623 - type_sparse_categorical_accuracy: 6.8259e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 339s 11s/step - loss: 0.5689 - price_loss: 0.5689 - type_loss: 3.1778 - price_sparse_categorical_accuracy: 0.7623 - type_sparse_categorical_accuracy: 6.8259e-04 - val_loss: 0.6859 - val_price_loss: 0.6859 - val_type_loss: 3.1776 - val_price_sparse_categorical_accuracy: 0.6992 - val_type_sparse_categorical_accuracy: 6.8213e-04
Epoch 33/50
30/30 [==============================] - ETA: 0s - loss: 0.5611 - price_loss: 0.5611 - type_loss: 3.1776 - price_sparse_categorical_accuracy: 0.7626 - type_sparse_categorical_accuracy: 8.5324e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 339s 11s/step - loss: 0.5611 - price_loss: 0.5611 - type_loss: 3.1776 - price_sparse_categorical_accuracy: 0.7626 - type_sparse_categorical_accuracy: 8.5324e-04 - val_loss: 0.6851 - val_price_loss: 0.6851 - val_type_loss: 3.1776 - val_price_sparse_categorical_accuracy: 0.7019 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 34/50
30/30 [==============================] - ETA: 0s - loss: 0.5531 - price_loss: 0.5531 - type_loss: 3.1777 - price_sparse_categorical_accuracy: 0.7686 - type_sparse_categorical_accuracy: 8.5324e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 339s 11s/step - loss: 0.5531 - price_loss: 0.5531 - type_loss: 3.1777 - price_sparse_categorical_accuracy: 0.7686 - type_sparse_categorical_accuracy: 8.5324e-04 - val_loss: 0.6854 - val_price_loss: 0.6854 - val_type_loss: 3.1776 - val_price_sparse_categorical_accuracy: 0.6992 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 35/50
30/30 [==============================] - ETA: 0s - loss: 0.5452 - price_loss: 0.5452 - type_loss: 3.1775 - price_sparse_categorical_accuracy: 0.7717 - type_sparse_categorical_accuracy: 0.0010 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 338s 11s/step - loss: 0.5452 - price_loss: 0.5452 - type_loss: 3.1775 - price_sparse_categorical_accuracy: 0.7717 - type_sparse_categorical_accuracy: 0.0010 - val_loss: 0.6850 - val_price_loss: 0.6850 - val_type_loss: 3.1775 - val_price_sparse_categorical_accuracy: 0.7012 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 36/50
30/30 [==============================] - ETA: 0s - loss: 0.5378 - price_loss: 0.5378 - type_loss: 3.1775 - price_sparse_categorical_accuracy: 0.7775 - type_sparse_categorical_accuracy: 0.0010 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 338s 11s/step - loss: 0.5378 - price_loss: 0.5378 - type_loss: 3.1775 - price_sparse_categorical_accuracy: 0.7775 - type_sparse_categorical_accuracy: 0.0010 - val_loss: 0.6853 - val_price_loss: 0.6853 - val_type_loss: 3.1774 - val_price_sparse_categorical_accuracy: 0.6978 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 37/50
30/30 [==============================] - ETA: 0s - loss: 0.5297 - price_loss: 0.5297 - type_loss: 3.1774 - price_sparse_categorical_accuracy: 0.7812 - type_sparse_categorical_accuracy: 0.0010 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 338s 11s/step - loss: 0.5297 - price_loss: 0.5297 - type_loss: 3.1774 - price_sparse_categorical_accuracy: 0.7812 - type_sparse_categorical_accuracy: 0.0010 - val_loss: 0.6863 - val_price_loss: 0.6863 - val_type_loss: 3.1774 - val_price_sparse_categorical_accuracy: 0.6930 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 38/50
30/30 [==============================] - ETA: 0s - loss: 0.5233 - price_loss: 0.5233 - type_loss: 3.1774 - price_sparse_categorical_accuracy: 0.7865 - type_sparse_categorical_accuracy: 0.0015 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 338s 11s/step - loss: 0.5233 - price_loss: 0.5233 - type_loss: 3.1774 - price_sparse_categorical_accuracy: 0.7865 - type_sparse_categorical_accuracy: 0.0015 - val_loss: 0.6853 - val_price_loss: 0.6853 - val_type_loss: 3.1773 - val_price_sparse_categorical_accuracy: 0.6992 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 39/50
30/30 [==============================] - ETA: 0s - loss: 0.5158 - price_loss: 0.5158 - type_loss: 3.1774 - price_sparse_categorical_accuracy: 0.7874 - type_sparse_categorical_accuracy: 0.0015 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 338s 11s/step - loss: 0.5158 - price_loss: 0.5158 - type_loss: 3.1774 - price_sparse_categorical_accuracy: 0.7874 - type_sparse_categorical_accuracy: 0.0015 - val_loss: 0.6856 - val_price_loss: 0.6856 - val_type_loss: 3.1772 - val_price_sparse_categorical_accuracy: 0.7019 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 40/50
30/30 [==============================] - ETA: 0s - loss: 0.5083 - price_loss: 0.5083 - type_loss: 3.1772 - price_sparse_categorical_accuracy: 0.7922 - type_sparse_categorical_accuracy: 0.0017 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 339s 11s/step - loss: 0.5083 - price_loss: 0.5083 - type_loss: 3.1772 - price_sparse_categorical_accuracy: 0.7922 - type_sparse_categorical_accuracy: 0.0017 - val_loss: 0.6872 - val_price_loss: 0.6872 - val_type_loss: 3.1771 - val_price_sparse_categorical_accuracy: 0.7005 - val_type_sparse_categorical_accuracy: 0.0014
plot_loss_accuracy_curves(history_model_2nd_trail)
________________________________________________________________________________________________________________________________________________________________________________________________________
Observation
Submission(model_2nd_trail , "2nd_trail")
model_save(model_2nd_trail , "2nd_trail")
230/230 [==============================] - 65s 282ms/step probabilities of predicted price [[0.82867235 0.14869116 0.02263645] [0.88663894 0.09319116 0.02016987] [0.8862957 0.09220586 0.02149838] ... [0.8225065 0.15670532 0.02078809] [0.98772705 0.01009632 0.00217664] [0.77818334 0.195048 0.02676866]] ____________________________________________________________________________________________________ values of of predicted price : [0 0 0 ... 0 0 0] counts of classes predicted in submitted file <bound method DataFrame.value_counts of id price 0 0 0 1 1 0 2 2 0 3 3 0 4 4 0 ... ... ... 7355 7626 1 7356 7627 0 7357 7628 0 7358 7629 0 7359 7630 0 [7360 rows x 2 columns]>
Observation
#clear session
tf.keras.backend.clear_session()
In this trail, we will
we expect that this changes will improve the performance
In this trail, we handled:
Text part:
We used Embedding layer enables us to convert each word into a fixed length vector of defined size. The resultant vector is a dense one with having real values instead of just 0's and 1's. The fixed length of word vectors helps us to represent words in a better way along with reduced dimensions.
Then we used an LSTM layer and vary the number of units (16)
Image part:
we used convolutional neural network layer with 64 filters, and a 5x5 mask, no padding , one stride with relu as activation function .
Then we used max pooling layer with a 2x2 mask,it reduced the size to 62x62x64
Followed by another convolutional neural network layer with 64 filters, and a 7x7 mask, no padding , one stride and drop out layer follow it .
Then we used max pooling layer with a 2x2 mask,it reduced the size to 28x28x64
Another convolutional neural network layer ,dropout and maxpooling layer
After that, we used flatten layer to flatten the multi-dimensional input tensors into a single dimension that it was (1152, ).
averaged_3_1rd_trail = text_model('lstm')
flattened_3_1rd_trail = image_model(conv2d =True ,drop = True)
model_3_1rd_trail = build_model(averaged_3_1rd_trail , flattened_3_1rd_trail , 1 , 0 , weights_focal = False)
history_model_3_1rd_trail , model_3_1rd_trail = fit_model(model_3_1rd_trail)
Model: "model"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_2 (InputLayer) [(None, 128, 128, 2 0 []
)]
conv2d_1 (Conv2D) (None, 124, 124, 64 3264 ['input_2[0][0]']
)
dropout (Dropout) (None, 124, 124, 64 0 ['conv2d_1[0][0]']
)
max_pooling2d (MaxPooling2D) (None, 62, 62, 64) 0 ['dropout[0][0]']
conv2d_2 (Conv2D) (None, 56, 56, 64) 200768 ['max_pooling2d[0][0]']
dropout_1 (Dropout) (None, 56, 56, 64) 0 ['conv2d_2[0][0]']
max_pooling2d_1 (MaxPooling2D) (None, 28, 28, 64) 0 ['dropout_1[0][0]']
conv2d_3 (Conv2D) (None, 13, 13, 32) 524320 ['max_pooling2d_1[0][0]']
input_1 (InputLayer) [(None, 177)] 0 []
dropout_2 (Dropout) (None, 13, 13, 32) 0 ['conv2d_3[0][0]']
embedding_1 (Embedding) (None, 177, 177) 7080000 ['input_1[0][0]']
max_pooling2d_2 (MaxPooling2D) (None, 6, 6, 32) 0 ['dropout_2[0][0]']
lstm_1 (LSTM) (None, 16) 12416 ['embedding_1[0][0]']
flatten (Flatten) (None, 1152) 0 ['max_pooling2d_2[0][0]']
tf.concat (TFOpLambda) (None, 1168) 0 ['lstm_1[0][0]',
'flatten[0][0]']
price (Dense) (None, 3) 3507 ['tf.concat[0][0]']
type (Dense) (None, 24) 28056 ['tf.concat[0][0]']
==================================================================================================
Total params: 7,852,331
Trainable params: 7,852,331
Non-trainable params: 0
__________________________________________________________________________________________________
Epoch 1/50
30/30 [==============================] - ETA: 0s - loss: 109.0499 - price_loss: 109.0499 - type_loss: 224.1192 - price_sparse_categorical_accuracy: 0.4923 - type_sparse_categorical_accuracy: 0.0222WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 148s 5s/step - loss: 109.0499 - price_loss: 109.0499 - type_loss: 224.1192 - price_sparse_categorical_accuracy: 0.4923 - type_sparse_categorical_accuracy: 0.0222 - val_loss: 1.0337 - val_price_loss: 1.0337 - val_type_loss: 3.2654 - val_price_sparse_categorical_accuracy: 0.5061 - val_type_sparse_categorical_accuracy: 0.0225
Epoch 2/50
30/30 [==============================] - ETA: 0s - loss: 0.9434 - price_loss: 0.9434 - type_loss: 3.9744 - price_sparse_categorical_accuracy: 0.5696 - type_sparse_categorical_accuracy: 0.0135WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 146s 5s/step - loss: 0.9434 - price_loss: 0.9434 - type_loss: 3.9744 - price_sparse_categorical_accuracy: 0.5696 - type_sparse_categorical_accuracy: 0.0135 - val_loss: 0.9025 - val_price_loss: 0.9025 - val_type_loss: 3.3906 - val_price_sparse_categorical_accuracy: 0.6235 - val_type_sparse_categorical_accuracy: 0.0055
Epoch 3/50
30/30 [==============================] - ETA: 0s - loss: 0.8623 - price_loss: 0.8623 - type_loss: 3.6804 - price_sparse_categorical_accuracy: 0.6094 - type_sparse_categorical_accuracy: 0.0075WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 144s 5s/step - loss: 0.8623 - price_loss: 0.8623 - type_loss: 3.6804 - price_sparse_categorical_accuracy: 0.6094 - type_sparse_categorical_accuracy: 0.0075 - val_loss: 0.8598 - val_price_loss: 0.8598 - val_type_loss: 3.2652 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0075
Epoch 4/50
30/30 [==============================] - ETA: 0s - loss: 0.8386 - price_loss: 0.8386 - type_loss: 3.4722 - price_sparse_categorical_accuracy: 0.6191 - type_sparse_categorical_accuracy: 0.0111WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 144s 5s/step - loss: 0.8386 - price_loss: 0.8386 - type_loss: 3.4722 - price_sparse_categorical_accuracy: 0.6191 - type_sparse_categorical_accuracy: 0.0111 - val_loss: 0.8367 - val_price_loss: 0.8367 - val_type_loss: 3.2334 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0150
Epoch 5/50
30/30 [==============================] - ETA: 0s - loss: 0.8392 - price_loss: 0.8392 - type_loss: 3.2320 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0189WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 144s 5s/step - loss: 0.8392 - price_loss: 0.8392 - type_loss: 3.2320 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0189 - val_loss: 0.8309 - val_price_loss: 0.8309 - val_type_loss: 3.1659 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0225
Epoch 6/50
30/30 [==============================] - ETA: 0s - loss: 0.8378 - price_loss: 0.8378 - type_loss: 3.1760 - price_sparse_categorical_accuracy: 0.6198 - type_sparse_categorical_accuracy: 0.0224WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 142s 5s/step - loss: 0.8378 - price_loss: 0.8378 - type_loss: 3.1760 - price_sparse_categorical_accuracy: 0.6198 - type_sparse_categorical_accuracy: 0.0224 - val_loss: 0.8322 - val_price_loss: 0.8322 - val_type_loss: 3.1651 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0225
Epoch 7/50
30/30 [==============================] - ETA: 0s - loss: 0.8373 - price_loss: 0.8373 - type_loss: 3.1738 - price_sparse_categorical_accuracy: 0.6198 - type_sparse_categorical_accuracy: 0.0224WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 142s 5s/step - loss: 0.8373 - price_loss: 0.8373 - type_loss: 3.1738 - price_sparse_categorical_accuracy: 0.6198 - type_sparse_categorical_accuracy: 0.0224 - val_loss: 0.8318 - val_price_loss: 0.8318 - val_type_loss: 3.1655 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0225
Epoch 8/50
30/30 [==============================] - ETA: 0s - loss: 0.8383 - price_loss: 0.8383 - type_loss: 3.1743 - price_sparse_categorical_accuracy: 0.6198 - type_sparse_categorical_accuracy: 0.0224WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 143s 5s/step - loss: 0.8383 - price_loss: 0.8383 - type_loss: 3.1743 - price_sparse_categorical_accuracy: 0.6198 - type_sparse_categorical_accuracy: 0.0224 - val_loss: 0.8322 - val_price_loss: 0.8322 - val_type_loss: 3.1663 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0225
Epoch 9/50
30/30 [==============================] - ETA: 0s - loss: 0.8382 - price_loss: 0.8382 - type_loss: 3.1757 - price_sparse_categorical_accuracy: 0.6198 - type_sparse_categorical_accuracy: 0.0224WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 142s 5s/step - loss: 0.8382 - price_loss: 0.8382 - type_loss: 3.1757 - price_sparse_categorical_accuracy: 0.6198 - type_sparse_categorical_accuracy: 0.0224 - val_loss: 0.8325 - val_price_loss: 0.8325 - val_type_loss: 3.1665 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0225
Epoch 10/50
30/30 [==============================] - ETA: 0s - loss: 0.8373 - price_loss: 0.8373 - type_loss: 3.1775 - price_sparse_categorical_accuracy: 0.6198 - type_sparse_categorical_accuracy: 0.0224WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 143s 5s/step - loss: 0.8373 - price_loss: 0.8373 - type_loss: 3.1775 - price_sparse_categorical_accuracy: 0.6198 - type_sparse_categorical_accuracy: 0.0224 - val_loss: 0.8311 - val_price_loss: 0.8311 - val_type_loss: 3.1684 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0225
plot_loss_accuracy_curves(history_model_3_1rd_trail)
________________________________________________________________________________________________________________________________________________________________________________________________________
Observation
Submission(model_3_1rd_trail , "3_1rd_trail")
model_save(model_3_1rd_trail , "3_1rd_trail")
230/230 [==============================] - 37s 161ms/step probabilities of predicted price [[0.61192906 0.30847278 0.07959815] [0.611068 0.30770263 0.08122937] [0.611068 0.30770263 0.08122937] ... [0.61131084 0.3078012 0.080888 ] [0.61293405 0.3070181 0.08004776] [0.61101705 0.3079689 0.0810141 ]] ____________________________________________________________________________________________________ values of of predicted price : [0 0 0 ... 0 0 0] counts of classes predicted in submitted file <bound method DataFrame.value_counts of id price 0 0 0 1 1 0 2 2 0 3 3 0 4 4 0 ... ... ... 7355 7626 0 7356 7627 0 7357 7628 0 7358 7629 0 7359 7630 0 [7360 rows x 2 columns]>
Observation
#clear session
tf.keras.backend.clear_session()
In this trail, we will
In this trail, we handled:
Text part:
We used Embedding layer enables us to convert each word into a fixed length vector of defined size. The resultant vector is a dense one with having real values instead of just 0's and 1's. The fixed length of word vectors helps us to represent words in a better way along with reduced dimensions.
Then we used an LSTM layer and vary the number of units (16)
Image part:
we used convolutional neural network layer with 64 filters, and a 5x5 mask, no padding , one stride with relu as activation function .
Then we used max pooling layer with a 2x2 mask,it reduced the size to 62x62x64
Followed by another convolutional neural network layer with 32 filters, and a 7x7 mask, no padding , one stride and drop out layer follow it .
Then we used max pooling layer with a 2x2 mask,it reduced the size to 23x23x32
averaged_3rd_trail = text_model('lstm')
flattened_3rd_trail = image_model(drop = True)
model_3rd_trail = build_model(averaged_3rd_trail , flattened_3rd_trail , 1 , 0 , weights_focal = False)
history_model_3rd_trail , model_3rd_trail = fit_model(model_3rd_trail)
Model: "model"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_2 (InputLayer) [(None, 128, 128, 2 0 []
)]
conv2d (Conv2D) (None, 124, 124, 64 3264 ['input_2[0][0]']
)
dropout (Dropout) (None, 124, 124, 64 0 ['conv2d[0][0]']
)
max_pooling2d (MaxPooling2D) (None, 62, 62, 64) 0 ['dropout[0][0]']
conv2d_1 (Conv2D) (None, 47, 47, 32) 524320 ['max_pooling2d[0][0]']
input_1 (InputLayer) [(None, 177)] 0 []
dropout_1 (Dropout) (None, 47, 47, 32) 0 ['conv2d_1[0][0]']
embedding (Embedding) (None, 177, 177) 7080000 ['input_1[0][0]']
max_pooling2d_1 (MaxPooling2D) (None, 23, 23, 32) 0 ['dropout_1[0][0]']
lstm (LSTM) (None, 16) 12416 ['embedding[0][0]']
flatten (Flatten) (None, 16928) 0 ['max_pooling2d_1[0][0]']
tf.concat (TFOpLambda) (None, 16944) 0 ['lstm[0][0]',
'flatten[0][0]']
price (Dense) (None, 3) 50835 ['tf.concat[0][0]']
type (Dense) (None, 24) 406680 ['tf.concat[0][0]']
==================================================================================================
Total params: 8,077,515
Trainable params: 8,077,515
Non-trainable params: 0
__________________________________________________________________________________________________
Epoch 1/50
30/30 [==============================] - ETA: 0s - loss: 92.1940 - price_loss: 92.1940 - type_loss: 19.0766 - price_sparse_categorical_accuracy: 0.4980 - type_sparse_categorical_accuracy: 0.0410 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 386s 13s/step - loss: 92.1940 - price_loss: 92.1940 - type_loss: 19.0766 - price_sparse_categorical_accuracy: 0.4980 - type_sparse_categorical_accuracy: 0.0410 - val_loss: 1.0460 - val_price_loss: 1.0460 - val_type_loss: 3.1736 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0000e+00
Epoch 2/50
30/30 [==============================] - ETA: 0s - loss: 0.9519 - price_loss: 0.9519 - type_loss: 3.1765 - price_sparse_categorical_accuracy: 0.6137 - type_sparse_categorical_accuracy: 0.0502 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 378s 13s/step - loss: 0.9519 - price_loss: 0.9519 - type_loss: 3.1765 - price_sparse_categorical_accuracy: 0.6137 - type_sparse_categorical_accuracy: 0.0502 - val_loss: 0.8606 - val_price_loss: 0.8606 - val_type_loss: 3.1610 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0000e+00
Epoch 3/50
30/30 [==============================] - ETA: 0s - loss: 0.8472 - price_loss: 0.8472 - type_loss: 3.1601 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 376s 13s/step - loss: 0.8472 - price_loss: 0.8472 - type_loss: 3.1601 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.8301 - val_price_loss: 0.8301 - val_type_loss: 3.1593 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0000e+00
Epoch 4/50
30/30 [==============================] - ETA: 0s - loss: 0.9463 - price_loss: 0.9463 - type_loss: 3.2655 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 6.8259e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 373s 12s/step - loss: 0.9463 - price_loss: 0.9463 - type_loss: 3.2655 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 6.8259e-04 - val_loss: 0.8306 - val_price_loss: 0.8306 - val_type_loss: 3.1593 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0000e+00
Epoch 5/50
30/30 [==============================] - ETA: 0s - loss: 0.8381 - price_loss: 0.8381 - type_loss: 3.1596 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 375s 12s/step - loss: 0.8381 - price_loss: 0.8381 - type_loss: 3.1596 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.8294 - val_price_loss: 0.8294 - val_type_loss: 3.1592 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0000e+00
Epoch 6/50
30/30 [==============================] - ETA: 0s - loss: 0.8376 - price_loss: 0.8376 - type_loss: 3.1596 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 373s 12s/step - loss: 0.8376 - price_loss: 0.8376 - type_loss: 3.1596 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.8295 - val_price_loss: 0.8295 - val_type_loss: 3.1592 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0000e+00
Epoch 7/50
30/30 [==============================] - ETA: 0s - loss: 0.8378 - price_loss: 0.8378 - type_loss: 3.1596 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 372s 12s/step - loss: 0.8378 - price_loss: 0.8378 - type_loss: 3.1596 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.8288 - val_price_loss: 0.8288 - val_type_loss: 3.1592 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0000e+00
Epoch 8/50
30/30 [==============================] - ETA: 0s - loss: 0.8376 - price_loss: 0.8376 - type_loss: 3.1596 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 372s 12s/step - loss: 0.8376 - price_loss: 0.8376 - type_loss: 3.1596 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.8285 - val_price_loss: 0.8285 - val_type_loss: 3.1592 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0000e+00
Epoch 9/50
30/30 [==============================] - ETA: 0s - loss: 0.8376 - price_loss: 0.8376 - type_loss: 3.1595 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 373s 12s/step - loss: 0.8376 - price_loss: 0.8376 - type_loss: 3.1595 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.8286 - val_price_loss: 0.8286 - val_type_loss: 3.1592 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0000e+00
Epoch 10/50
30/30 [==============================] - ETA: 0s - loss: 0.8376 - price_loss: 0.8376 - type_loss: 3.1596 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 373s 12s/step - loss: 0.8376 - price_loss: 0.8376 - type_loss: 3.1596 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.8286 - val_price_loss: 0.8286 - val_type_loss: 3.1592 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0000e+00
Epoch 11/50
30/30 [==============================] - ETA: 0s - loss: 0.8376 - price_loss: 0.8376 - type_loss: 3.1595 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 372s 12s/step - loss: 0.8376 - price_loss: 0.8376 - type_loss: 3.1595 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.8286 - val_price_loss: 0.8286 - val_type_loss: 3.1592 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0000e+00
Epoch 12/50
30/30 [==============================] - ETA: 0s - loss: 0.8375 - price_loss: 0.8375 - type_loss: 3.1595 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 373s 12s/step - loss: 0.8375 - price_loss: 0.8375 - type_loss: 3.1595 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.8288 - val_price_loss: 0.8288 - val_type_loss: 3.1592 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0000e+00
Epoch 13/50
30/30 [==============================] - ETA: 0s - loss: 0.8375 - price_loss: 0.8375 - type_loss: 3.1596 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 383s 13s/step - loss: 0.8375 - price_loss: 0.8375 - type_loss: 3.1596 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.8288 - val_price_loss: 0.8288 - val_type_loss: 3.1592 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0000e+00
plot_loss_accuracy_curves(history_model_3rd_trail)
________________________________________________________________________________________________________________________________________________________________________________________________________
Observation
Submission(model_3rd_trail , "3rd_trail")
model_save(model_3rd_trail , "3rd_trail")
Observation
#clear session
tf.keras.backend.clear_session()
In this trail, we will
In this trail, we handled:
Text part:
We used Embedding layer enables us to convert each word into a fixed length vector of defined size. The resultant vector is a dense one with having real values instead of just 0's and 1's. The fixed length of word vectors helps us to represent words in a better way along with reduced dimensions.
Then we used an GRU layer and vary the number of units
Image part:
we used convolutional neural network layer with 64 filters, and a 5x5 mask, no padding , one stride with relu as activation function .
Then we used max pooling layer with a 2x2 mask,it reduced the size to 62x62x64
Followed by another convolutional neural network layer with 32 filters, and a 7x7 mask, no padding , one stride
Then we used max pooling layer with a 2x2 mask,it reduced the size to 23x23x32
averaged_4th_trail = text_model('gru')
flattened_4th_trail = image_model()
model_4th_trail = build_model(averaged_4th_trail , flattened_4th_trail , 1 , 0 , weights_focal = False)
history_model_4th_trail , model_4th_trail = fit_model(model_4th_trail)
Model: "model_2"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_2 (InputLayer) [(None, 128, 128, 2 0 []
)]
conv2d_5 (Conv2D) (None, 124, 124, 64 3264 ['input_2[0][0]']
)
max_pooling2d_5 (MaxPooling2D) (None, 62, 62, 64) 0 ['conv2d_5[0][0]']
input_1 (InputLayer) [(None, 177)] 0 []
conv2d_6 (Conv2D) (None, 47, 47, 32) 524320 ['max_pooling2d_5[0][0]']
embedding_2 (Embedding) (None, 177, 177) 7080000 ['input_1[0][0]']
max_pooling2d_6 (MaxPooling2D) (None, 23, 23, 32) 0 ['conv2d_6[0][0]']
gru_1 (GRU) (None, 7) 3906 ['embedding_2[0][0]']
flatten_2 (Flatten) (None, 16928) 0 ['max_pooling2d_6[0][0]']
tf.concat_2 (TFOpLambda) (None, 16935) 0 ['gru_1[0][0]',
'flatten_2[0][0]']
price (Dense) (None, 3) 50808 ['tf.concat_2[0][0]']
type (Dense) (None, 24) 406464 ['tf.concat_2[0][0]']
==================================================================================================
Total params: 8,068,762
Trainable params: 8,068,762
Non-trainable params: 0
__________________________________________________________________________________________________
Epoch 1/50
30/30 [==============================] - ETA: 0s - loss: 117.0614 - price_loss: 117.0614 - type_loss: 51.2422 - price_sparse_categorical_accuracy: 0.5602 - type_sparse_categorical_accuracy: 0.0099 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 346s 11s/step - loss: 117.0614 - price_loss: 117.0614 - type_loss: 51.2422 - price_sparse_categorical_accuracy: 0.5602 - type_sparse_categorical_accuracy: 0.0099 - val_loss: 1.0701 - val_price_loss: 1.0701 - val_type_loss: 3.1810 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 2/50
30/30 [==============================] - ETA: 0s - loss: 1.0134 - price_loss: 1.0134 - type_loss: 3.1876 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0020 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 345s 12s/step - loss: 1.0134 - price_loss: 1.0134 - type_loss: 3.1876 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0020 - val_loss: 0.9339 - val_price_loss: 0.9339 - val_type_loss: 3.1993 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 3/50
30/30 [==============================] - ETA: 0s - loss: 0.8942 - price_loss: 0.8942 - type_loss: 3.2049 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0020 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 344s 11s/step - loss: 0.8942 - price_loss: 0.8942 - type_loss: 3.2049 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0020 - val_loss: 0.8584 - val_price_loss: 0.8584 - val_type_loss: 3.2091 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 4/50
30/30 [==============================] - ETA: 0s - loss: 0.8535 - price_loss: 0.8535 - type_loss: 3.2091 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0020 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 346s 12s/step - loss: 0.8535 - price_loss: 0.8535 - type_loss: 3.2091 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0020 - val_loss: 0.8396 - val_price_loss: 0.8396 - val_type_loss: 3.2104 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 5/50
30/30 [==============================] - ETA: 0s - loss: 0.8430 - price_loss: 0.8430 - type_loss: 3.2097 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0020 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 343s 11s/step - loss: 0.8430 - price_loss: 0.8430 - type_loss: 3.2097 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0020 - val_loss: 0.8333 - val_price_loss: 0.8333 - val_type_loss: 3.2107 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 6/50
30/30 [==============================] - ETA: 0s - loss: 0.8395 - price_loss: 0.8395 - type_loss: 3.2100 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0020 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 344s 11s/step - loss: 0.8395 - price_loss: 0.8395 - type_loss: 3.2100 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0020 - val_loss: 0.8310 - val_price_loss: 0.8310 - val_type_loss: 3.2108 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 7/50
30/30 [==============================] - ETA: 0s - loss: 0.8381 - price_loss: 0.8381 - type_loss: 3.2101 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0020 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 344s 11s/step - loss: 0.8381 - price_loss: 0.8381 - type_loss: 3.2101 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0020 - val_loss: 0.8300 - val_price_loss: 0.8300 - val_type_loss: 3.2109 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 8/50
30/30 [==============================] - ETA: 0s - loss: 0.8377 - price_loss: 0.8377 - type_loss: 3.2101 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0020 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 344s 11s/step - loss: 0.8377 - price_loss: 0.8377 - type_loss: 3.2101 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0020 - val_loss: 0.8293 - val_price_loss: 0.8293 - val_type_loss: 3.2110 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 9/50
30/30 [==============================] - ETA: 0s - loss: 0.8375 - price_loss: 0.8375 - type_loss: 3.2102 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0020 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 344s 11s/step - loss: 0.8375 - price_loss: 0.8375 - type_loss: 3.2102 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0020 - val_loss: 0.8291 - val_price_loss: 0.8291 - val_type_loss: 3.2110 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 10/50
30/30 [==============================] - ETA: 0s - loss: 0.8373 - price_loss: 0.8373 - type_loss: 3.2102 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0020 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 344s 11s/step - loss: 0.8373 - price_loss: 0.8373 - type_loss: 3.2102 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0020 - val_loss: 0.8289 - val_price_loss: 0.8289 - val_type_loss: 3.2110 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 11/50
30/30 [==============================] - ETA: 0s - loss: 0.8373 - price_loss: 0.8373 - type_loss: 3.2102 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0020 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 343s 11s/step - loss: 0.8373 - price_loss: 0.8373 - type_loss: 3.2102 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0020 - val_loss: 0.8288 - val_price_loss: 0.8288 - val_type_loss: 3.2110 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 12/50
30/30 [==============================] - ETA: 0s - loss: 0.8373 - price_loss: 0.8373 - type_loss: 3.2102 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0020 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 342s 11s/step - loss: 0.8373 - price_loss: 0.8373 - type_loss: 3.2102 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0020 - val_loss: 0.8286 - val_price_loss: 0.8286 - val_type_loss: 3.2110 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 13/50
30/30 [==============================] - ETA: 0s - loss: 0.8373 - price_loss: 0.8373 - type_loss: 3.2102 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0020 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 343s 11s/step - loss: 0.8373 - price_loss: 0.8373 - type_loss: 3.2102 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0020 - val_loss: 0.8286 - val_price_loss: 0.8286 - val_type_loss: 3.2110 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 14/50
30/30 [==============================] - ETA: 0s - loss: 0.8373 - price_loss: 0.8373 - type_loss: 3.2102 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0020 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 343s 11s/step - loss: 0.8373 - price_loss: 0.8373 - type_loss: 3.2102 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0020 - val_loss: 0.8286 - val_price_loss: 0.8286 - val_type_loss: 3.2110 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 15/50
30/30 [==============================] - ETA: 0s - loss: 0.8372 - price_loss: 0.8372 - type_loss: 3.2102 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0020 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 344s 11s/step - loss: 0.8372 - price_loss: 0.8372 - type_loss: 3.2102 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0020 - val_loss: 0.8287 - val_price_loss: 0.8287 - val_type_loss: 3.2110 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 16/50
30/30 [==============================] - ETA: 0s - loss: 0.8372 - price_loss: 0.8372 - type_loss: 3.2102 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0020 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 343s 11s/step - loss: 0.8372 - price_loss: 0.8372 - type_loss: 3.2102 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0020 - val_loss: 0.8287 - val_price_loss: 0.8287 - val_type_loss: 3.2110 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 17/50
30/30 [==============================] - ETA: 0s - loss: 0.8370 - price_loss: 0.8370 - type_loss: 3.2102 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0020 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 345s 12s/step - loss: 0.8370 - price_loss: 0.8370 - type_loss: 3.2102 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0020 - val_loss: 0.8288 - val_price_loss: 0.8288 - val_type_loss: 3.2111 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 18/50
30/30 [==============================] - ETA: 0s - loss: 0.8371 - price_loss: 0.8371 - type_loss: 3.2102 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0020 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 342s 11s/step - loss: 0.8371 - price_loss: 0.8371 - type_loss: 3.2102 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0020 - val_loss: 0.8287 - val_price_loss: 0.8287 - val_type_loss: 3.2111 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 19/50
30/30 [==============================] - ETA: 0s - loss: 0.8371 - price_loss: 0.8371 - type_loss: 3.2102 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0020 WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 347s 12s/step - loss: 0.8371 - price_loss: 0.8371 - type_loss: 3.2102 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0020 - val_loss: 0.8287 - val_price_loss: 0.8287 - val_type_loss: 3.2111 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
plot_loss_accuracy_curves(history_model_4th_trail)
________________________________________________________________________________________________________________________________________________________________________________________________________
Observation
Submission(model_4th_trail , "4th_trail")
model_save(model_4th_trail , "4th_trail")
230/230 [==============================] - 73s 315ms/step probabilities of predicted price [[0.62022805 0.31507713 0.06469464] [0.6202283 0.31507707 0.06469455] [0.6202283 0.3150771 0.06469458] ... [0.6202283 0.31507707 0.06469455] [0.62022746 0.3150776 0.06469501] [0.6202273 0.3150776 0.06469508]] ____________________________________________________________________________________________________ values of of predicted price : [0 0 0 ... 0 0 0] counts of classes predicted in submitted file <bound method DataFrame.value_counts of id price 0 0 0 1 1 0 2 2 0 3 3 0 4 4 0 ... ... ... 7355 7626 0 7356 7627 0 7357 7628 0 7358 7629 0 7359 7630 0 [7360 rows x 2 columns]>
Observation
#clear session
tf.keras.backend.clear_session()
In this trail, we will
In text part :
In Image part:
we expect that this changes will improve the performance
In this trail, we handled:
Text part:
We used Embedding layer enables us to convert each word into a fixed length vector of defined size. The resultant vector is a dense one with having real values instead of just 0's and 1's. The fixed length of word vectors helps us to represent words in a better way along with reduced dimensions.
Then we used abi-directional lstm layer and vary the number of units (10)
Image part:
we used convolutional neural network layer with 64 filters, and a 5x5 mask, no padding , one stride with relu as activation function .
Then we used max pooling layer with a 2x2 mask,it reduced the size to 62x62x64
Followed by another convolutional neural network layer with 64 filters, and a 7x7 mask, no padding , one stride and drop out layer follow it .
Then we used max pooling layer with a 2x2 mask,it reduced the size to 28x28x64
Another convolutional neural network layer ,dropout and maxpooling layer
After that, we used flatten layer to flatten the multi-dimensional input tensors into a single dimension that it was (1152, ).
from tensorflow.keras.layers import Conv2D, Flatten, Dense, MaxPool2D, Dropout
averaged_5th_trail = text_model('bidirect')
flattened_5th_trail = image_model(drop = True , conv2d =True)
model_5th_trail = build_model(averaged_5th_trail , flattened_5th_trail , 1 , 0 , weights_focal = True)
history_model_5th_trail , model_5th_trail = fit_model(model_5th_trail)
Model: "model"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_2 (InputLayer) [(None, 128, 128, 2 0 []
)]
conv2d (Conv2D) (None, 124, 124, 64 3264 ['input_2[0][0]']
)
dropout (Dropout) (None, 124, 124, 64 0 ['conv2d[0][0]']
)
max_pooling2d (MaxPooling2D) (None, 62, 62, 64) 0 ['dropout[0][0]']
conv2d_1 (Conv2D) (None, 56, 56, 64) 200768 ['max_pooling2d[0][0]']
dropout_1 (Dropout) (None, 56, 56, 64) 0 ['conv2d_1[0][0]']
max_pooling2d_1 (MaxPooling2D) (None, 28, 28, 64) 0 ['dropout_1[0][0]']
conv2d_2 (Conv2D) (None, 13, 13, 32) 524320 ['max_pooling2d_1[0][0]']
input_1 (InputLayer) [(None, 177)] 0 []
dropout_2 (Dropout) (None, 13, 13, 32) 0 ['conv2d_2[0][0]']
embedding_2 (Embedding) (None, 177, 177) 7080000 ['input_1[0][0]']
max_pooling2d_2 (MaxPooling2D) (None, 6, 6, 32) 0 ['dropout_2[0][0]']
bidirectional (Bidirectional) (None, 10) 7320 ['embedding_2[0][0]']
flatten (Flatten) (None, 1152) 0 ['max_pooling2d_2[0][0]']
tf.concat (TFOpLambda) (None, 1162) 0 ['bidirectional[0][0]',
'flatten[0][0]']
price (Dense) (None, 3) 3489 ['tf.concat[0][0]']
type (Dense) (None, 24) 27912 ['tf.concat[0][0]']
==================================================================================================
Total params: 7,847,073
Trainable params: 7,847,073
Non-trainable params: 0
__________________________________________________________________________________________________
Epoch 1/50
30/30 [==============================] - ETA: 0s - loss: 6.2138 - price_loss: 6.2138 - type_loss: 16.1113 - price_sparse_categorical_accuracy: 0.6133 - type_sparse_categorical_accuracy: 1.7065e-04WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 170s 6s/step - loss: 6.2138 - price_loss: 6.2138 - type_loss: 16.1113 - price_sparse_categorical_accuracy: 0.6133 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 6.0580 - val_price_loss: 6.0580 - val_type_loss: 16.1181 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0000e+00
Epoch 2/50
30/30 [==============================] - ETA: 0s - loss: 6.1309 - price_loss: 6.1309 - type_loss: 16.1123 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 166s 6s/step - loss: 6.1309 - price_loss: 6.1309 - type_loss: 16.1123 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 6.0580 - val_price_loss: 6.0580 - val_type_loss: 16.1181 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0000e+00
Epoch 3/50
30/30 [==============================] - ETA: 0s - loss: 6.1309 - price_loss: 6.1309 - type_loss: 16.1121 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 166s 6s/step - loss: 6.1309 - price_loss: 6.1309 - type_loss: 16.1121 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 6.0580 - val_price_loss: 6.0580 - val_type_loss: 16.1181 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0000e+00
Epoch 4/50
30/30 [==============================] - ETA: 0s - loss: 6.1309 - price_loss: 6.1309 - type_loss: 16.1121 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 164s 5s/step - loss: 6.1309 - price_loss: 6.1309 - type_loss: 16.1121 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 6.0580 - val_price_loss: 6.0580 - val_type_loss: 16.1181 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0000e+00
Epoch 5/50
30/30 [==============================] - ETA: 0s - loss: 6.1309 - price_loss: 6.1309 - type_loss: 16.1123 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 166s 6s/step - loss: 6.1309 - price_loss: 6.1309 - type_loss: 16.1123 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 6.0580 - val_price_loss: 6.0580 - val_type_loss: 16.1181 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0000e+00
Epoch 6/50
30/30 [==============================] - ETA: 0s - loss: 6.1309 - price_loss: 6.1309 - type_loss: 16.1122 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 167s 6s/step - loss: 6.1309 - price_loss: 6.1309 - type_loss: 16.1122 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 6.0580 - val_price_loss: 6.0580 - val_type_loss: 16.1181 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0000e+00
plot_loss_accuracy_curves(history_model_5th_trail)
________________________________________________________________________________________________________________________________________________________________________________________________________
Observation
we expect that this trail will achieve score on kaggle same as the previous trails
Submission(model_5th_trail , "5th_trail")
model_save(model_5th_trail , "5th_trail")
230/230 [==============================] - 26s 111ms/step probabilities of predicted price [[1.0000000e+00 0.0000000e+00 0.0000000e+00] [1.0000000e+00 0.0000000e+00 0.0000000e+00] [9.9218994e-01 9.1849722e-04 6.8914564e-03] ... [9.9219090e-01 9.2125707e-04 6.8878522e-03] [9.9220324e-01 9.1885030e-04 6.8779546e-03] [9.9217039e-01 9.2139794e-04 6.9082417e-03]] ____________________________________________________________________________________________________ values of of predicted price : [0 0 0 ... 0 0 0] counts of classes predicted in submitted file <bound method DataFrame.value_counts of id price 0 0 0 1 1 0 2 2 0 3 3 0 4 4 0 ... ... ... 7355 7626 0 7356 7627 0 7357 7628 0 7358 7629 0 7359 7630 0 [7360 rows x 2 columns]>
Observation
#clear session
tf.keras.backend.clear_session()
In this trail, we will
In text part :
In Image part:
we expect that this changes will improve the performance
In this trail, we handled:
Text part:
We used Embedding layer enables us to convert each word into a fixed length vector of defined size. The resultant vector is a dense one with having real values instead of just 0's and 1's. The fixed length of word vectors helps us to represent words in a better way along with reduced dimensions.
Image part:
this time, we will predict two outputs (type and price )
def Resize_image_pretrained(filepath):
try:
# Resize original images into 48 x 48 x 3
image = Image.open(filepath).resize((48,48))
arr = np.array(image)
# Check if there is an image is grayscale and conert it into rgb and resize it
if len(arr.shape)!= 3 :
print("GrayScale Image" , filepath)
image = Image.open(filepath).convert('RGB').resize((48,48))
arr = np.array(image)
except:
print(filepath)
arr = np.zeros((48, 48, 3))
return arr
# Resize train images into 48 x48 x 3
x_train_image = np.array([Resize_image_pretrained(i) for i in data.image])
/kaggle/input/copy-of-cisc-873-dm-w23-a4/a4/img_train/4176.jpg GrayScale Image /kaggle/input/copy-of-cisc-873-dm-w23-a4/a4/img_train/5878.jpg /kaggle/input/copy-of-cisc-873-dm-w23-a4/a4/img_train/6083.jpg
# Resize test images into 48 x48 x 3 same as train images
x_test_image = np.array([Resize_image_pretrained(i) for i in submission_data['image']])
/kaggle/input/copy-of-cisc-873-dm-w23-a4/a4/img_test/7314.jpg
# reshape your input and output
x_train_text_id = np.array(x_train_text_id, dtype = 'float32')
in_text = keras.Input(batch_shape=(None, max_len))
in_image = keras.Input(batch_shape=(None,48,48,3))
checkpoint_filepath = '/tmp/checkpoint'
#----------------------Image part --------------------------
base_model = tf.keras.applications.VGG16(input_shape=(48,48,3),include_top=False,weights='imagenet')(in_image)
flattened_pretrained = Flatten()(base_model)
# --------------------- Text Part ------------------------
averaged_pretrained = text_model('embedding')
# ------------------ Build Model --------------------
model_pretrained_trail = build_model(averaged_pretrained , flattened_pretrained , 0.5 , 0.5 , weights_focal = False)
history_model_pretrained_trail , model_pretrained_trail = fit_model(model_pretrained_trail)
Model: "model_1"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_15 (InputLayer) [(None, 177)] 0 []
input_16 (InputLayer) [(None, 48, 48, 3)] 0 []
embedding_1 (Embedding) (None, 177, 177) 7080000 ['input_15[0][0]']
vgg16 (Functional) (None, 1, 1, 512) 14714688 ['input_16[0][0]']
tf.math.reduce_mean_1 (TFOpLam (None, 177) 0 ['embedding_1[0][0]']
bda)
flatten_2 (Flatten) (None, 512) 0 ['vgg16[0][0]']
tf.concat_1 (TFOpLambda) (None, 689) 0 ['tf.math.reduce_mean_1[0][0]',
'flatten_2[0][0]']
price (Dense) (None, 3) 2070 ['tf.concat_1[0][0]']
type (Dense) (None, 24) 16560 ['tf.concat_1[0][0]']
==================================================================================================
Total params: 21,813,318
Trainable params: 21,813,318
Non-trainable params: 0
__________________________________________________________________________________________________
Epoch 1/50
30/30 [==============================] - 20s 282ms/step - loss: 20.7955 - price_loss: 17.2519 - type_loss: 24.3392 - price_sparse_categorical_accuracy: 0.5683 - type_sparse_categorical_accuracy: 0.6323 - val_loss: 0.9435 - val_price_loss: 0.8411 - val_type_loss: 1.0459 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.7688
Epoch 2/50
30/30 [==============================] - 8s 254ms/step - loss: 0.9613 - price_loss: 0.8504 - type_loss: 1.0721 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.7560 - val_loss: 0.9088 - val_price_loss: 0.8338 - val_type_loss: 0.9837 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.7688
Epoch 3/50
30/30 [==============================] - 6s 184ms/step - loss: 0.9363 - price_loss: 0.8436 - type_loss: 1.0290 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.7560 - val_loss: 0.9029 - val_price_loss: 0.8317 - val_type_loss: 0.9741 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.7688
Epoch 4/50
30/30 [==============================] - 7s 217ms/step - loss: 0.9301 - price_loss: 0.8389 - type_loss: 1.0213 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.7560 - val_loss: 0.9074 - val_price_loss: 0.8271 - val_type_loss: 0.9876 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.7688
Epoch 5/50
30/30 [==============================] - 5s 187ms/step - loss: 0.9247 - price_loss: 0.8343 - type_loss: 1.0151 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.7560 - val_loss: 0.8992 - val_price_loss: 0.8265 - val_type_loss: 0.9719 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.7688
Epoch 6/50
30/30 [==============================] - 6s 189ms/step - loss: 0.9186 - price_loss: 0.8293 - type_loss: 1.0079 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.7560 - val_loss: 0.8895 - val_price_loss: 0.8174 - val_type_loss: 0.9617 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.7688
Epoch 7/50
30/30 [==============================] - 5s 158ms/step - loss: 0.9115 - price_loss: 0.8213 - type_loss: 1.0016 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.7560 - val_loss: 0.8842 - val_price_loss: 0.8102 - val_type_loss: 0.9582 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.7688
Epoch 8/50
30/30 [==============================] - 5s 166ms/step - loss: 0.9033 - price_loss: 0.8116 - type_loss: 0.9949 - price_sparse_categorical_accuracy: 0.6227 - type_sparse_categorical_accuracy: 0.7560 - val_loss: 0.8810 - val_price_loss: 0.8070 - val_type_loss: 0.9550 - val_price_sparse_categorical_accuracy: 0.6310 - val_type_sparse_categorical_accuracy: 0.7688
Epoch 9/50
30/30 [==============================] - 4s 147ms/step - loss: 0.8943 - price_loss: 0.8002 - type_loss: 0.9884 - price_sparse_categorical_accuracy: 0.6273 - type_sparse_categorical_accuracy: 0.7560 - val_loss: 0.8690 - val_price_loss: 0.7919 - val_type_loss: 0.9462 - val_price_sparse_categorical_accuracy: 0.6323 - val_type_sparse_categorical_accuracy: 0.7688
Epoch 10/50
30/30 [==============================] - 4s 150ms/step - loss: 0.8836 - price_loss: 0.7865 - type_loss: 0.9806 - price_sparse_categorical_accuracy: 0.6314 - type_sparse_categorical_accuracy: 0.7560 - val_loss: 0.8615 - val_price_loss: 0.7827 - val_type_loss: 0.9403 - val_price_sparse_categorical_accuracy: 0.6351 - val_type_sparse_categorical_accuracy: 0.7688
Epoch 11/50
30/30 [==============================] - 4s 135ms/step - loss: 0.8707 - price_loss: 0.7715 - type_loss: 0.9700 - price_sparse_categorical_accuracy: 0.6418 - type_sparse_categorical_accuracy: 0.7560 - val_loss: 0.8490 - val_price_loss: 0.7677 - val_type_loss: 0.9302 - val_price_sparse_categorical_accuracy: 0.6412 - val_type_sparse_categorical_accuracy: 0.7688
Epoch 12/50
30/30 [==============================] - 4s 140ms/step - loss: 0.8571 - price_loss: 0.7573 - type_loss: 0.9568 - price_sparse_categorical_accuracy: 0.6529 - type_sparse_categorical_accuracy: 0.7561 - val_loss: 0.8479 - val_price_loss: 0.7741 - val_type_loss: 0.9217 - val_price_sparse_categorical_accuracy: 0.6808 - val_type_sparse_categorical_accuracy: 0.7688
Epoch 13/50
30/30 [==============================] - 4s 124ms/step - loss: 0.8471 - price_loss: 0.7468 - type_loss: 0.9475 - price_sparse_categorical_accuracy: 0.6604 - type_sparse_categorical_accuracy: 0.7561 - val_loss: 0.8336 - val_price_loss: 0.7487 - val_type_loss: 0.9186 - val_price_sparse_categorical_accuracy: 0.6658 - val_type_sparse_categorical_accuracy: 0.7688
Epoch 14/50
30/30 [==============================] - 4s 145ms/step - loss: 0.8315 - price_loss: 0.7294 - type_loss: 0.9336 - price_sparse_categorical_accuracy: 0.6735 - type_sparse_categorical_accuracy: 0.7561 - val_loss: 0.8212 - val_price_loss: 0.7399 - val_type_loss: 0.9024 - val_price_sparse_categorical_accuracy: 0.6719 - val_type_sparse_categorical_accuracy: 0.7688
Epoch 15/50
30/30 [==============================] - 4s 144ms/step - loss: 0.8173 - price_loss: 0.7160 - type_loss: 0.9185 - price_sparse_categorical_accuracy: 0.6809 - type_sparse_categorical_accuracy: 0.7561 - val_loss: 0.8135 - val_price_loss: 0.7306 - val_type_loss: 0.8964 - val_price_sparse_categorical_accuracy: 0.6739 - val_type_sparse_categorical_accuracy: 0.7688
Epoch 16/50
30/30 [==============================] - 4s 123ms/step - loss: 0.8071 - price_loss: 0.7058 - type_loss: 0.9085 - price_sparse_categorical_accuracy: 0.6828 - type_sparse_categorical_accuracy: 0.7573 - val_loss: 0.8039 - val_price_loss: 0.7247 - val_type_loss: 0.8831 - val_price_sparse_categorical_accuracy: 0.6733 - val_type_sparse_categorical_accuracy: 0.7688
Epoch 17/50
30/30 [==============================] - 4s 140ms/step - loss: 0.7940 - price_loss: 0.6928 - type_loss: 0.8952 - price_sparse_categorical_accuracy: 0.6973 - type_sparse_categorical_accuracy: 0.7582 - val_loss: 0.8024 - val_price_loss: 0.7221 - val_type_loss: 0.8828 - val_price_sparse_categorical_accuracy: 0.6835 - val_type_sparse_categorical_accuracy: 0.7694
Epoch 18/50
30/30 [==============================] - 4s 119ms/step - loss: 0.7836 - price_loss: 0.6849 - type_loss: 0.8824 - price_sparse_categorical_accuracy: 0.6991 - type_sparse_categorical_accuracy: 0.7587 - val_loss: 0.7923 - val_price_loss: 0.7167 - val_type_loss: 0.8678 - val_price_sparse_categorical_accuracy: 0.6705 - val_type_sparse_categorical_accuracy: 0.7701
Epoch 19/50
30/30 [==============================] - 3s 113ms/step - loss: 0.7717 - price_loss: 0.6731 - type_loss: 0.8703 - price_sparse_categorical_accuracy: 0.7041 - type_sparse_categorical_accuracy: 0.7594 - val_loss: 0.7861 - val_price_loss: 0.7129 - val_type_loss: 0.8593 - val_price_sparse_categorical_accuracy: 0.6855 - val_type_sparse_categorical_accuracy: 0.7708
Epoch 20/50
30/30 [==============================] - 3s 110ms/step - loss: 0.7600 - price_loss: 0.6637 - type_loss: 0.8563 - price_sparse_categorical_accuracy: 0.7121 - type_sparse_categorical_accuracy: 0.7614 - val_loss: 0.7821 - val_price_loss: 0.7057 - val_type_loss: 0.8584 - val_price_sparse_categorical_accuracy: 0.6835 - val_type_sparse_categorical_accuracy: 0.7715
Epoch 21/50
30/30 [==============================] - 4s 125ms/step - loss: 0.7460 - price_loss: 0.6509 - type_loss: 0.8410 - price_sparse_categorical_accuracy: 0.7195 - type_sparse_categorical_accuracy: 0.7623 - val_loss: 0.7743 - val_price_loss: 0.7034 - val_type_loss: 0.8451 - val_price_sparse_categorical_accuracy: 0.6855 - val_type_sparse_categorical_accuracy: 0.7756
Epoch 22/50
30/30 [==============================] - 3s 98ms/step - loss: 0.7335 - price_loss: 0.6411 - type_loss: 0.8258 - price_sparse_categorical_accuracy: 0.7234 - type_sparse_categorical_accuracy: 0.7626 - val_loss: 0.7715 - val_price_loss: 0.7033 - val_type_loss: 0.8398 - val_price_sparse_categorical_accuracy: 0.6808 - val_type_sparse_categorical_accuracy: 0.7742
Epoch 23/50
30/30 [==============================] - 3s 105ms/step - loss: 0.7230 - price_loss: 0.6337 - type_loss: 0.8122 - price_sparse_categorical_accuracy: 0.7273 - type_sparse_categorical_accuracy: 0.7650 - val_loss: 0.7648 - val_price_loss: 0.6972 - val_type_loss: 0.8323 - val_price_sparse_categorical_accuracy: 0.6869 - val_type_sparse_categorical_accuracy: 0.7763
Epoch 24/50
30/30 [==============================] - 4s 125ms/step - loss: 0.7111 - price_loss: 0.6235 - type_loss: 0.7987 - price_sparse_categorical_accuracy: 0.7328 - type_sparse_categorical_accuracy: 0.7688 - val_loss: 0.7641 - val_price_loss: 0.7020 - val_type_loss: 0.8262 - val_price_sparse_categorical_accuracy: 0.6965 - val_type_sparse_categorical_accuracy: 0.7810
Epoch 25/50
30/30 [==============================] - 3s 102ms/step - loss: 0.7013 - price_loss: 0.6159 - type_loss: 0.7867 - price_sparse_categorical_accuracy: 0.7403 - type_sparse_categorical_accuracy: 0.7701 - val_loss: 0.7555 - val_price_loss: 0.6925 - val_type_loss: 0.8185 - val_price_sparse_categorical_accuracy: 0.6985 - val_type_sparse_categorical_accuracy: 0.7810
Epoch 26/50
30/30 [==============================] - 3s 99ms/step - loss: 0.6892 - price_loss: 0.6070 - type_loss: 0.7713 - price_sparse_categorical_accuracy: 0.7459 - type_sparse_categorical_accuracy: 0.7746 - val_loss: 0.7545 - val_price_loss: 0.6960 - val_type_loss: 0.8129 - val_price_sparse_categorical_accuracy: 0.6903 - val_type_sparse_categorical_accuracy: 0.7810
Epoch 27/50
30/30 [==============================] - 4s 119ms/step - loss: 0.6784 - price_loss: 0.5987 - type_loss: 0.7581 - price_sparse_categorical_accuracy: 0.7490 - type_sparse_categorical_accuracy: 0.7758 - val_loss: 0.7479 - val_price_loss: 0.6889 - val_type_loss: 0.8069 - val_price_sparse_categorical_accuracy: 0.6999 - val_type_sparse_categorical_accuracy: 0.7810
Epoch 28/50
30/30 [==============================] - 4s 121ms/step - loss: 0.6685 - price_loss: 0.5914 - type_loss: 0.7455 - price_sparse_categorical_accuracy: 0.7538 - type_sparse_categorical_accuracy: 0.7828 - val_loss: 0.7450 - val_price_loss: 0.6886 - val_type_loss: 0.8014 - val_price_sparse_categorical_accuracy: 0.7012 - val_type_sparse_categorical_accuracy: 0.7831
Epoch 29/50
30/30 [==============================] - 4s 119ms/step - loss: 0.6600 - price_loss: 0.5854 - type_loss: 0.7346 - price_sparse_categorical_accuracy: 0.7609 - type_sparse_categorical_accuracy: 0.7852 - val_loss: 0.7465 - val_price_loss: 0.6963 - val_type_loss: 0.7967 - val_price_sparse_categorical_accuracy: 0.6903 - val_type_sparse_categorical_accuracy: 0.7831
Epoch 30/50
30/30 [==============================] - 3s 115ms/step - loss: 0.6489 - price_loss: 0.5775 - type_loss: 0.7203 - price_sparse_categorical_accuracy: 0.7630 - type_sparse_categorical_accuracy: 0.7906 - val_loss: 0.7423 - val_price_loss: 0.6908 - val_type_loss: 0.7939 - val_price_sparse_categorical_accuracy: 0.6944 - val_type_sparse_categorical_accuracy: 0.7851
Epoch 31/50
30/30 [==============================] - 3s 116ms/step - loss: 0.6391 - price_loss: 0.5701 - type_loss: 0.7081 - price_sparse_categorical_accuracy: 0.7657 - type_sparse_categorical_accuracy: 0.7951 - val_loss: 0.7371 - val_price_loss: 0.6863 - val_type_loss: 0.7880 - val_price_sparse_categorical_accuracy: 0.6985 - val_type_sparse_categorical_accuracy: 0.7831
Epoch 32/50
30/30 [==============================] - 3s 101ms/step - loss: 0.6294 - price_loss: 0.5638 - type_loss: 0.6950 - price_sparse_categorical_accuracy: 0.7683 - type_sparse_categorical_accuracy: 0.7974 - val_loss: 0.7364 - val_price_loss: 0.6874 - val_type_loss: 0.7853 - val_price_sparse_categorical_accuracy: 0.6965 - val_type_sparse_categorical_accuracy: 0.7844
Epoch 33/50
30/30 [==============================] - 3s 101ms/step - loss: 0.6192 - price_loss: 0.5553 - type_loss: 0.6832 - price_sparse_categorical_accuracy: 0.7735 - type_sparse_categorical_accuracy: 0.8007 - val_loss: 0.7374 - val_price_loss: 0.6947 - val_type_loss: 0.7801 - val_price_sparse_categorical_accuracy: 0.6937 - val_type_sparse_categorical_accuracy: 0.7844
Epoch 34/50
30/30 [==============================] - 3s 106ms/step - loss: 0.6115 - price_loss: 0.5520 - type_loss: 0.6710 - price_sparse_categorical_accuracy: 0.7724 - type_sparse_categorical_accuracy: 0.8036 - val_loss: 0.7287 - val_price_loss: 0.6854 - val_type_loss: 0.7720 - val_price_sparse_categorical_accuracy: 0.6992 - val_type_sparse_categorical_accuracy: 0.7851
Epoch 35/50
30/30 [==============================] - 3s 104ms/step - loss: 0.6033 - price_loss: 0.5451 - type_loss: 0.6615 - price_sparse_categorical_accuracy: 0.7766 - type_sparse_categorical_accuracy: 0.8051 - val_loss: 0.7295 - val_price_loss: 0.6881 - val_type_loss: 0.7709 - val_price_sparse_categorical_accuracy: 0.6985 - val_type_sparse_categorical_accuracy: 0.7885
Epoch 36/50
30/30 [==============================] - 3s 114ms/step - loss: 0.5947 - price_loss: 0.5394 - type_loss: 0.6501 - price_sparse_categorical_accuracy: 0.7792 - type_sparse_categorical_accuracy: 0.8092 - val_loss: 0.7268 - val_price_loss: 0.6871 - val_type_loss: 0.7666 - val_price_sparse_categorical_accuracy: 0.6978 - val_type_sparse_categorical_accuracy: 0.7872
Epoch 37/50
30/30 [==============================] - 3s 102ms/step - loss: 0.5846 - price_loss: 0.5311 - type_loss: 0.6381 - price_sparse_categorical_accuracy: 0.7848 - type_sparse_categorical_accuracy: 0.8102 - val_loss: 0.7313 - val_price_loss: 0.6939 - val_type_loss: 0.7686 - val_price_sparse_categorical_accuracy: 0.6924 - val_type_sparse_categorical_accuracy: 0.7892
Epoch 38/50
30/30 [==============================] - 3s 102ms/step - loss: 0.5783 - price_loss: 0.5274 - type_loss: 0.6291 - price_sparse_categorical_accuracy: 0.7874 - type_sparse_categorical_accuracy: 0.8160 - val_loss: 0.7254 - val_price_loss: 0.6887 - val_type_loss: 0.7621 - val_price_sparse_categorical_accuracy: 0.6958 - val_type_sparse_categorical_accuracy: 0.7906
Epoch 39/50
30/30 [==============================] - 4s 118ms/step - loss: 0.5691 - price_loss: 0.5196 - type_loss: 0.6186 - price_sparse_categorical_accuracy: 0.7891 - type_sparse_categorical_accuracy: 0.8183 - val_loss: 0.7262 - val_price_loss: 0.6968 - val_type_loss: 0.7556 - val_price_sparse_categorical_accuracy: 0.6917 - val_type_sparse_categorical_accuracy: 0.7933
plot_loss_accuracy_curves(history_model_pretrained_trail)
________________________________________________________________________________________________________________________________________________________________________________________________________
Observation
Submission(model_pretrained_trail , "pretrained_model" )
model_save(model_pretrained_trail , "pretrained_model")
230/230 [==============================] - 2s 7ms/step probabilities of predicted price [[0.8716053 0.10324236 0.02515234] [0.92313784 0.0640364 0.01282569] [0.9494431 0.03879583 0.01176111] ... [0.86276793 0.11867621 0.01855591] [0.9877437 0.00911802 0.00313827] [0.7879836 0.18346207 0.02855434]] ____________________________________________________________________________________________________ values of of predicted price : [0 0 0 ... 0 0 0] counts of classes predicted in submitted file <bound method DataFrame.value_counts of id price 0 0 0 1 1 0 2 2 0 3 3 0 4 4 0 ... ... ... 7355 7626 1 7356 7627 0 7357 7628 0 7358 7629 0 7359 7630 0 [7360 rows x 2 columns]>
Observation
Embeddig on text and simple Convolution without focal loss (2nd trail ) achieved high Score compared to other trails (68.342%)
pretrained model is the second rank in the highest Score
Dropout and focal loss have a good impact in preventing overfitting
averaged_lstm = text_model('lstm')
model_lstm_text_only = build_model_one_input(averaged_lstm,in_text, 1 , 0 , weights_focal = False)
history_model_lstm_text_only, model_lstm_text_only = fit_model_one_input(model_lstm_text_only, x_train_text_id)
Model: "model_4"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_1 (InputLayer) [(None, 177)] 0 []
embedding_8 (Embedding) (None, 177, 177) 7080000 ['input_1[0][0]']
lstm_5 (LSTM) (None, 16) 12416 ['embedding_8[0][0]']
price (Dense) (None, 3) 51 ['lstm_5[0][0]']
type (Dense) (None, 24) 408 ['lstm_5[0][0]']
==================================================================================================
Total params: 7,092,875
Trainable params: 7,092,875
Non-trainable params: 0
__________________________________________________________________________________________________
Epoch 1/50
30/30 [==============================] - ETA: 0s - loss: 0.9349 - price_loss: 0.9349 - type_loss: 3.0953 - price_sparse_categorical_accuracy: 0.6024 - type_sparse_categorical_accuracy: 0.0036WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 5s 115ms/step - loss: 0.9349 - price_loss: 0.9349 - type_loss: 3.0953 - price_sparse_categorical_accuracy: 0.6024 - type_sparse_categorical_accuracy: 0.0036 - val_loss: 0.8317 - val_price_loss: 0.8317 - val_type_loss: 3.0481 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0041
Epoch 2/50
30/30 [==============================] - ETA: 0s - loss: 0.8390 - price_loss: 0.8390 - type_loss: 3.0572 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0044WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 3s 105ms/step - loss: 0.8390 - price_loss: 0.8390 - type_loss: 3.0572 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0044 - val_loss: 0.8292 - val_price_loss: 0.8292 - val_type_loss: 3.0564 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0041
Epoch 3/50
30/30 [==============================] - ETA: 0s - loss: 0.8373 - price_loss: 0.8373 - type_loss: 3.0628 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0044WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 3s 106ms/step - loss: 0.8373 - price_loss: 0.8373 - type_loss: 3.0628 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0044 - val_loss: 0.8326 - val_price_loss: 0.8326 - val_type_loss: 3.0669 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0041
Epoch 4/50
30/30 [==============================] - ETA: 0s - loss: 0.8397 - price_loss: 0.8397 - type_loss: 3.0689 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0044WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 3s 104ms/step - loss: 0.8397 - price_loss: 0.8397 - type_loss: 3.0689 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0044 - val_loss: 0.8291 - val_price_loss: 0.8291 - val_type_loss: 3.0677 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0041
Epoch 5/50
30/30 [==============================] - ETA: 0s - loss: 0.8385 - price_loss: 0.8385 - type_loss: 3.0736 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0044WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 3s 106ms/step - loss: 0.8385 - price_loss: 0.8385 - type_loss: 3.0736 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0044 - val_loss: 0.8294 - val_price_loss: 0.8294 - val_type_loss: 3.0771 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0041
Epoch 6/50
30/30 [==============================] - ETA: 0s - loss: 0.8381 - price_loss: 0.8381 - type_loss: 3.0787 - price_sparse_categorical_accuracy: 0.6198 - type_sparse_categorical_accuracy: 0.0044WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 3s 104ms/step - loss: 0.8381 - price_loss: 0.8381 - type_loss: 3.0787 - price_sparse_categorical_accuracy: 0.6198 - type_sparse_categorical_accuracy: 0.0044 - val_loss: 0.8317 - val_price_loss: 0.8317 - val_type_loss: 3.0844 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0041
Epoch 7/50
30/30 [==============================] - ETA: 0s - loss: 0.8378 - price_loss: 0.8378 - type_loss: 3.0863 - price_sparse_categorical_accuracy: 0.6198 - type_sparse_categorical_accuracy: 0.0044WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 3s 105ms/step - loss: 0.8378 - price_loss: 0.8378 - type_loss: 3.0863 - price_sparse_categorical_accuracy: 0.6198 - type_sparse_categorical_accuracy: 0.0044 - val_loss: 0.8294 - val_price_loss: 0.8294 - val_type_loss: 3.0794 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0041
Epoch 8/50
30/30 [==============================] - ETA: 0s - loss: 0.8381 - price_loss: 0.8381 - type_loss: 3.0850 - price_sparse_categorical_accuracy: 0.6198 - type_sparse_categorical_accuracy: 0.0044WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 3s 103ms/step - loss: 0.8381 - price_loss: 0.8381 - type_loss: 3.0850 - price_sparse_categorical_accuracy: 0.6198 - type_sparse_categorical_accuracy: 0.0044 - val_loss: 0.8288 - val_price_loss: 0.8288 - val_type_loss: 3.0843 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0041
Epoch 9/50
30/30 [==============================] - ETA: 0s - loss: 0.8382 - price_loss: 0.8382 - type_loss: 3.0879 - price_sparse_categorical_accuracy: 0.6198 - type_sparse_categorical_accuracy: 0.0044WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 3s 104ms/step - loss: 0.8382 - price_loss: 0.8382 - type_loss: 3.0879 - price_sparse_categorical_accuracy: 0.6198 - type_sparse_categorical_accuracy: 0.0044 - val_loss: 0.8293 - val_price_loss: 0.8293 - val_type_loss: 3.0836 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0041
Epoch 10/50
30/30 [==============================] - ETA: 0s - loss: 0.8382 - price_loss: 0.8382 - type_loss: 3.0925 - price_sparse_categorical_accuracy: 0.6198 - type_sparse_categorical_accuracy: 0.0044WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 3s 110ms/step - loss: 0.8382 - price_loss: 0.8382 - type_loss: 3.0925 - price_sparse_categorical_accuracy: 0.6198 - type_sparse_categorical_accuracy: 0.0044 - val_loss: 0.8296 - val_price_loss: 0.8296 - val_type_loss: 3.0874 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0041
Epoch 11/50
30/30 [==============================] - ETA: 0s - loss: 0.8378 - price_loss: 0.8378 - type_loss: 3.0951 - price_sparse_categorical_accuracy: 0.6198 - type_sparse_categorical_accuracy: 0.0036WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 3s 109ms/step - loss: 0.8378 - price_loss: 0.8378 - type_loss: 3.0951 - price_sparse_categorical_accuracy: 0.6198 - type_sparse_categorical_accuracy: 0.0036 - val_loss: 0.8292 - val_price_loss: 0.8292 - val_type_loss: 3.0942 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0041
Epoch 12/50
30/30 [==============================] - ETA: 0s - loss: 0.8373 - price_loss: 0.8373 - type_loss: 3.0961 - price_sparse_categorical_accuracy: 0.6198 - type_sparse_categorical_accuracy: 0.0044WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 3s 108ms/step - loss: 0.8373 - price_loss: 0.8373 - type_loss: 3.0961 - price_sparse_categorical_accuracy: 0.6198 - type_sparse_categorical_accuracy: 0.0044 - val_loss: 0.8288 - val_price_loss: 0.8288 - val_type_loss: 3.0922 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0041
Epoch 13/50
30/30 [==============================] - ETA: 0s - loss: 0.8375 - price_loss: 0.8375 - type_loss: 3.0969 - price_sparse_categorical_accuracy: 0.6198 - type_sparse_categorical_accuracy: 0.0027WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 3s 111ms/step - loss: 0.8375 - price_loss: 0.8375 - type_loss: 3.0969 - price_sparse_categorical_accuracy: 0.6198 - type_sparse_categorical_accuracy: 0.0027 - val_loss: 0.8287 - val_price_loss: 0.8287 - val_type_loss: 3.0932 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0041
Epoch 14/50
30/30 [==============================] - ETA: 0s - loss: 0.8380 - price_loss: 0.8380 - type_loss: 3.0974 - price_sparse_categorical_accuracy: 0.6198 - type_sparse_categorical_accuracy: 0.0038WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 3s 112ms/step - loss: 0.8380 - price_loss: 0.8380 - type_loss: 3.0974 - price_sparse_categorical_accuracy: 0.6198 - type_sparse_categorical_accuracy: 0.0038 - val_loss: 0.8289 - val_price_loss: 0.8289 - val_type_loss: 3.0935 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 15/50
30/30 [==============================] - ETA: 0s - loss: 0.8379 - price_loss: 0.8379 - type_loss: 3.0981 - price_sparse_categorical_accuracy: 0.6198 - type_sparse_categorical_accuracy: 0.0032WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 3s 113ms/step - loss: 0.8379 - price_loss: 0.8379 - type_loss: 3.0981 - price_sparse_categorical_accuracy: 0.6198 - type_sparse_categorical_accuracy: 0.0032 - val_loss: 0.8290 - val_price_loss: 0.8290 - val_type_loss: 3.0970 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 16/50
30/30 [==============================] - ETA: 0s - loss: 0.8380 - price_loss: 0.8380 - type_loss: 3.1010 - price_sparse_categorical_accuracy: 0.6198 - type_sparse_categorical_accuracy: 0.0022WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 3s 110ms/step - loss: 0.8380 - price_loss: 0.8380 - type_loss: 3.1010 - price_sparse_categorical_accuracy: 0.6198 - type_sparse_categorical_accuracy: 0.0022 - val_loss: 0.8291 - val_price_loss: 0.8291 - val_type_loss: 3.1003 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 17/50
30/30 [==============================] - ETA: 0s - loss: 0.8374 - price_loss: 0.8374 - type_loss: 3.1024 - price_sparse_categorical_accuracy: 0.6198 - type_sparse_categorical_accuracy: 0.0014WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 3s 109ms/step - loss: 0.8374 - price_loss: 0.8374 - type_loss: 3.1024 - price_sparse_categorical_accuracy: 0.6198 - type_sparse_categorical_accuracy: 0.0014 - val_loss: 0.8288 - val_price_loss: 0.8288 - val_type_loss: 3.0978 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 18/50
30/30 [==============================] - ETA: 0s - loss: 0.8373 - price_loss: 0.8373 - type_loss: 3.1021 - price_sparse_categorical_accuracy: 0.6198 - type_sparse_categorical_accuracy: 0.0014WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 3s 112ms/step - loss: 0.8373 - price_loss: 0.8373 - type_loss: 3.1021 - price_sparse_categorical_accuracy: 0.6198 - type_sparse_categorical_accuracy: 0.0014 - val_loss: 0.8288 - val_price_loss: 0.8288 - val_type_loss: 3.0983 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
plot_loss_accuracy_curves(history_model_lstm_text_only)
________________________________________________________________________________________________________________________________________________________________________________________________________
submission_one_input(model_lstm_text_only , "lstm_text_only" ,x_test_text )
model_save(model_lstm_text_only , "lstm_text_only")
230/230 [==============================] - 2s 8ms/step probabilities of predicted price [[0.6278251 0.30912906 0.06304578] [0.6278251 0.30912906 0.0630458 ] [0.6278251 0.30912906 0.06304578] ... [0.6278251 0.30912906 0.0630458 ] [0.6278251 0.30912912 0.0630458 ] [0.6278251 0.30912912 0.0630458 ]] ____________________________________________________________________________________________________ values of of predicted price : [0 0 0 ... 0 0 0] counts of classes predicted in submitted file <bound method DataFrame.value_counts of id price 0 0 0 1 1 0 2 2 0 3 3 0 4 4 0 ... ... ... 7355 7626 0 7356 7627 0 7357 7628 0 7358 7629 0 7359 7630 0 [7360 rows x 2 columns]>
averaged_bidirect = text_model('bidirect')
model_bidirect_text_only = build_model_one_input(averaged_bidirect,in_text, 1 , 0 , weights_focal = False)
history_model_bidirect_text_only, model_bidirect_text_only = fit_model_one_input(model_bidirect_text_only,x_train_text_id)
Model: "model_6"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_1 (InputLayer) [(None, 177)] 0 []
embedding_10 (Embedding) (None, 177, 177) 7080000 ['input_1[0][0]']
bidirectional_3 (Bidirectional (None, 10) 7320 ['embedding_10[0][0]']
)
price (Dense) (None, 3) 33 ['bidirectional_3[0][0]']
type (Dense) (None, 24) 264 ['bidirectional_3[0][0]']
==================================================================================================
Total params: 7,087,617
Trainable params: 7,087,617
Non-trainable params: 0
__________________________________________________________________________________________________
Epoch 1/50
30/30 [==============================] - ETA: 0s - loss: 0.9722 - price_loss: 0.9722 - type_loss: 3.1322 - price_sparse_categorical_accuracy: 0.5867 - type_sparse_categorical_accuracy: 8.5324e-04WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 7s 157ms/step - loss: 0.9722 - price_loss: 0.9722 - type_loss: 3.1322 - price_sparse_categorical_accuracy: 0.5867 - type_sparse_categorical_accuracy: 8.5324e-04 - val_loss: 0.8533 - val_price_loss: 0.8533 - val_type_loss: 3.0910 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0000e+00
Epoch 2/50
30/30 [==============================] - ETA: 0s - loss: 0.7933 - price_loss: 0.7933 - type_loss: 3.0976 - price_sparse_categorical_accuracy: 0.6229 - type_sparse_categorical_accuracy: 3.4130e-04WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 4s 134ms/step - loss: 0.7933 - price_loss: 0.7933 - type_loss: 3.0976 - price_sparse_categorical_accuracy: 0.6229 - type_sparse_categorical_accuracy: 3.4130e-04 - val_loss: 0.7630 - val_price_loss: 0.7630 - val_type_loss: 3.1038 - val_price_sparse_categorical_accuracy: 0.6562 - val_type_sparse_categorical_accuracy: 0.0000e+00
Epoch 3/50
30/30 [==============================] - ETA: 0s - loss: 0.7108 - price_loss: 0.7108 - type_loss: 3.1309 - price_sparse_categorical_accuracy: 0.7053 - type_sparse_categorical_accuracy: 6.8259e-04WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 4s 133ms/step - loss: 0.7108 - price_loss: 0.7108 - type_loss: 3.1309 - price_sparse_categorical_accuracy: 0.7053 - type_sparse_categorical_accuracy: 6.8259e-04 - val_loss: 0.7404 - val_price_loss: 0.7404 - val_type_loss: 3.1348 - val_price_sparse_categorical_accuracy: 0.6664 - val_type_sparse_categorical_accuracy: 0.0000e+00
Epoch 4/50
30/30 [==============================] - ETA: 0s - loss: 0.6450 - price_loss: 0.6450 - type_loss: 3.1389 - price_sparse_categorical_accuracy: 0.7611 - type_sparse_categorical_accuracy: 6.8259e-04WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 4s 133ms/step - loss: 0.6450 - price_loss: 0.6450 - type_loss: 3.1389 - price_sparse_categorical_accuracy: 0.7611 - type_sparse_categorical_accuracy: 6.8259e-04 - val_loss: 0.7386 - val_price_loss: 0.7386 - val_type_loss: 3.1342 - val_price_sparse_categorical_accuracy: 0.6535 - val_type_sparse_categorical_accuracy: 0.0000e+00
Epoch 5/50
30/30 [==============================] - ETA: 0s - loss: 0.5927 - price_loss: 0.5927 - type_loss: 3.1515 - price_sparse_categorical_accuracy: 0.7879 - type_sparse_categorical_accuracy: 3.4130e-04WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 4s 134ms/step - loss: 0.5927 - price_loss: 0.5927 - type_loss: 3.1515 - price_sparse_categorical_accuracy: 0.7879 - type_sparse_categorical_accuracy: 3.4130e-04 - val_loss: 0.7365 - val_price_loss: 0.7365 - val_type_loss: 3.1556 - val_price_sparse_categorical_accuracy: 0.6712 - val_type_sparse_categorical_accuracy: 0.0000e+00
Epoch 6/50
30/30 [==============================] - ETA: 0s - loss: 0.5411 - price_loss: 0.5411 - type_loss: 3.1699 - price_sparse_categorical_accuracy: 0.8116 - type_sparse_categorical_accuracy: 3.4130e-04WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 4s 133ms/step - loss: 0.5411 - price_loss: 0.5411 - type_loss: 3.1699 - price_sparse_categorical_accuracy: 0.8116 - type_sparse_categorical_accuracy: 3.4130e-04 - val_loss: 0.7789 - val_price_loss: 0.7789 - val_type_loss: 3.1661 - val_price_sparse_categorical_accuracy: 0.6248 - val_type_sparse_categorical_accuracy: 0.0000e+00
Epoch 7/50
30/30 [==============================] - ETA: 0s - loss: 0.5091 - price_loss: 0.5091 - type_loss: 3.1755 - price_sparse_categorical_accuracy: 0.8213 - type_sparse_categorical_accuracy: 3.4130e-04WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 4s 132ms/step - loss: 0.5091 - price_loss: 0.5091 - type_loss: 3.1755 - price_sparse_categorical_accuracy: 0.8213 - type_sparse_categorical_accuracy: 3.4130e-04 - val_loss: 0.7848 - val_price_loss: 0.7848 - val_type_loss: 3.1713 - val_price_sparse_categorical_accuracy: 0.6678 - val_type_sparse_categorical_accuracy: 0.0000e+00
Epoch 8/50
30/30 [==============================] - ETA: 0s - loss: 0.4746 - price_loss: 0.4746 - type_loss: 3.1898 - price_sparse_categorical_accuracy: 0.8374 - type_sparse_categorical_accuracy: 3.4130e-04WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 4s 132ms/step - loss: 0.4746 - price_loss: 0.4746 - type_loss: 3.1898 - price_sparse_categorical_accuracy: 0.8374 - type_sparse_categorical_accuracy: 3.4130e-04 - val_loss: 0.8129 - val_price_loss: 0.8129 - val_type_loss: 3.1852 - val_price_sparse_categorical_accuracy: 0.6583 - val_type_sparse_categorical_accuracy: 0.0000e+00
Epoch 9/50
30/30 [==============================] - ETA: 0s - loss: 0.4328 - price_loss: 0.4328 - type_loss: 3.2078 - price_sparse_categorical_accuracy: 0.8568 - type_sparse_categorical_accuracy: 3.4130e-04WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 4s 142ms/step - loss: 0.4328 - price_loss: 0.4328 - type_loss: 3.2078 - price_sparse_categorical_accuracy: 0.8568 - type_sparse_categorical_accuracy: 3.4130e-04 - val_loss: 0.8443 - val_price_loss: 0.8443 - val_type_loss: 3.1954 - val_price_sparse_categorical_accuracy: 0.6596 - val_type_sparse_categorical_accuracy: 0.0000e+00
Epoch 10/50
30/30 [==============================] - ETA: 0s - loss: 0.4067 - price_loss: 0.4067 - type_loss: 3.2097 - price_sparse_categorical_accuracy: 0.8640 - type_sparse_categorical_accuracy: 1.7065e-04WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 4s 139ms/step - loss: 0.4067 - price_loss: 0.4067 - type_loss: 3.2097 - price_sparse_categorical_accuracy: 0.8640 - type_sparse_categorical_accuracy: 1.7065e-04 - val_loss: 0.8702 - val_price_loss: 0.8702 - val_type_loss: 3.2036 - val_price_sparse_categorical_accuracy: 0.6398 - val_type_sparse_categorical_accuracy: 0.0000e+00
plot_loss_accuracy_curves(history_model_bidirect_text_only)
________________________________________________________________________________________________________________________________________________________________________________________________________
submission_one_input(model_bidirect_text_only , "bidirect_text_only" ,x_test_text )
model_save(model_bidirect_text_only , "bidirect_text_only")
230/230 [==============================] - 3s 9ms/step probabilities of predicted price [[0.92706954 0.05543373 0.01749679] [0.9402488 0.04247082 0.01728036] [0.9464477 0.03822266 0.0153297 ] ... [0.86352503 0.11578169 0.02069321] [0.9448001 0.03706077 0.01813912] [0.934714 0.04939555 0.01589049]] ____________________________________________________________________________________________________ values of of predicted price : [0 0 0 ... 0 0 0] counts of classes predicted in submitted file <bound method DataFrame.value_counts of id price 0 0 0 1 1 0 2 2 0 3 3 1 4 4 1 ... ... ... 7355 7626 1 7356 7627 1 7357 7628 0 7358 7629 0 7359 7630 0 [7360 rows x 2 columns]>
flatten_conv2d = image_model(conv2d=True)
model_conv2d_image_only = build_model_one_input(flatten_conv2d,in_image, 1 , 0 , weights_focal = False)
history_model_conv2d_image_only, model_conv2d_image_only = fit_model_one_input(model_conv2d_image_only, x_train_image)
Model: "model_7"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_2 (InputLayer) [(None, 128, 128, 2 0 []
)]
conv2d_9 (Conv2D) (None, 124, 124, 64 3264 ['input_2[0][0]']
)
max_pooling2d_9 (MaxPooling2D) (None, 62, 62, 64) 0 ['conv2d_9[0][0]']
conv2d_10 (Conv2D) (None, 56, 56, 64) 200768 ['max_pooling2d_9[0][0]']
max_pooling2d_10 (MaxPooling2D (None, 28, 28, 64) 0 ['conv2d_10[0][0]']
)
conv2d_11 (Conv2D) (None, 13, 13, 32) 524320 ['max_pooling2d_10[0][0]']
max_pooling2d_11 (MaxPooling2D (None, 6, 6, 32) 0 ['conv2d_11[0][0]']
)
flatten_3 (Flatten) (None, 1152) 0 ['max_pooling2d_11[0][0]']
price (Dense) (None, 3) 3459 ['flatten_3[0][0]']
type (Dense) (None, 24) 27672 ['flatten_3[0][0]']
==================================================================================================
Total params: 759,483
Trainable params: 759,483
Non-trainable params: 0
__________________________________________________________________________________________________
Epoch 1/50
30/30 [==============================] - ETA: 0s - loss: 27.0710 - price_loss: 27.0710 - type_loss: 33.7682 - price_sparse_categorical_accuracy: 0.5367 - type_sparse_categorical_accuracy: 0.0084WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 136s 5s/step - loss: 27.0710 - price_loss: 27.0710 - type_loss: 33.7682 - price_sparse_categorical_accuracy: 0.5367 - type_sparse_categorical_accuracy: 0.0084 - val_loss: 1.0760 - val_price_loss: 1.0760 - val_type_loss: 3.1765 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 6.8213e-04
Epoch 2/50
30/30 [==============================] - ETA: 0s - loss: 1.0534 - price_loss: 1.0534 - type_loss: 3.1763 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 8.5324e-04WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 138s 5s/step - loss: 1.0534 - price_loss: 1.0534 - type_loss: 3.1763 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 8.5324e-04 - val_loss: 1.0201 - val_price_loss: 1.0201 - val_type_loss: 3.1772 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 3/50
30/30 [==============================] - ETA: 0s - loss: 0.9798 - price_loss: 0.9798 - type_loss: 3.1783 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0017WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 137s 5s/step - loss: 0.9798 - price_loss: 0.9798 - type_loss: 3.1783 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0017 - val_loss: 0.9252 - val_price_loss: 0.9252 - val_type_loss: 3.1798 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 4/50
30/30 [==============================] - ETA: 0s - loss: 0.8913 - price_loss: 0.8913 - type_loss: 3.1810 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0017WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 137s 5s/step - loss: 0.8913 - price_loss: 0.8913 - type_loss: 3.1810 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0017 - val_loss: 0.8540 - val_price_loss: 0.8540 - val_type_loss: 3.1824 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 5/50
30/30 [==============================] - ETA: 0s - loss: 0.8479 - price_loss: 0.8479 - type_loss: 3.1832 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0017WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 137s 5s/step - loss: 0.8479 - price_loss: 0.8479 - type_loss: 3.1832 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0017 - val_loss: 0.8328 - val_price_loss: 0.8328 - val_type_loss: 3.1840 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 6/50
30/30 [==============================] - ETA: 0s - loss: 0.8385 - price_loss: 0.8385 - type_loss: 3.1842 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0017WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 136s 5s/step - loss: 0.8385 - price_loss: 0.8385 - type_loss: 3.1842 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0017 - val_loss: 0.8299 - val_price_loss: 0.8299 - val_type_loss: 3.1848 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 7/50
30/30 [==============================] - ETA: 0s - loss: 0.8377 - price_loss: 0.8377 - type_loss: 3.1848 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0017WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 136s 5s/step - loss: 0.8377 - price_loss: 0.8377 - type_loss: 3.1848 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0017 - val_loss: 0.8294 - val_price_loss: 0.8294 - val_type_loss: 3.1850 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 8/50
30/30 [==============================] - ETA: 0s - loss: 0.8377 - price_loss: 0.8377 - type_loss: 3.1848 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0017WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 136s 5s/step - loss: 0.8377 - price_loss: 0.8377 - type_loss: 3.1848 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0017 - val_loss: 0.8295 - val_price_loss: 0.8295 - val_type_loss: 3.1850 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 9/50
30/30 [==============================] - ETA: 0s - loss: 0.8376 - price_loss: 0.8376 - type_loss: 3.1849 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0017WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 137s 5s/step - loss: 0.8376 - price_loss: 0.8376 - type_loss: 3.1849 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0017 - val_loss: 0.8295 - val_price_loss: 0.8295 - val_type_loss: 3.1850 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 10/50
30/30 [==============================] - ETA: 0s - loss: 0.8375 - price_loss: 0.8375 - type_loss: 3.1848 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0017WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 137s 5s/step - loss: 0.8375 - price_loss: 0.8375 - type_loss: 3.1848 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0017 - val_loss: 0.8294 - val_price_loss: 0.8294 - val_type_loss: 3.1850 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 11/50
30/30 [==============================] - ETA: 0s - loss: 0.8376 - price_loss: 0.8376 - type_loss: 3.1849 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0017WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 137s 5s/step - loss: 0.8376 - price_loss: 0.8376 - type_loss: 3.1849 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0017 - val_loss: 0.8294 - val_price_loss: 0.8294 - val_type_loss: 3.1851 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 12/50
30/30 [==============================] - ETA: 0s - loss: 0.8375 - price_loss: 0.8375 - type_loss: 3.1848 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0017WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 137s 5s/step - loss: 0.8375 - price_loss: 0.8375 - type_loss: 3.1848 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0017 - val_loss: 0.8295 - val_price_loss: 0.8295 - val_type_loss: 3.1850 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 13/50
30/30 [==============================] - ETA: 0s - loss: 0.8378 - price_loss: 0.8378 - type_loss: 3.1850 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0017WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 137s 5s/step - loss: 0.8378 - price_loss: 0.8378 - type_loss: 3.1850 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0017 - val_loss: 0.8296 - val_price_loss: 0.8296 - val_type_loss: 3.1849 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 14/50
30/30 [==============================] - ETA: 0s - loss: 0.8378 - price_loss: 0.8378 - type_loss: 3.1848 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0017WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 136s 5s/step - loss: 0.8378 - price_loss: 0.8378 - type_loss: 3.1848 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0017 - val_loss: 0.8295 - val_price_loss: 0.8295 - val_type_loss: 3.1849 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 15/50
30/30 [==============================] - ETA: 0s - loss: 0.8376 - price_loss: 0.8376 - type_loss: 3.1850 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0017WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 136s 5s/step - loss: 0.8376 - price_loss: 0.8376 - type_loss: 3.1850 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0017 - val_loss: 0.8294 - val_price_loss: 0.8294 - val_type_loss: 3.1851 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 16/50
30/30 [==============================] - ETA: 0s - loss: 0.8377 - price_loss: 0.8377 - type_loss: 3.1848 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0017WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 137s 5s/step - loss: 0.8377 - price_loss: 0.8377 - type_loss: 3.1848 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0017 - val_loss: 0.8295 - val_price_loss: 0.8295 - val_type_loss: 3.1849 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 17/50
30/30 [==============================] - ETA: 0s - loss: 0.8377 - price_loss: 0.8377 - type_loss: 3.1850 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0017WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 137s 5s/step - loss: 0.8377 - price_loss: 0.8377 - type_loss: 3.1850 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0017 - val_loss: 0.8295 - val_price_loss: 0.8295 - val_type_loss: 3.1850 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 18/50
30/30 [==============================] - ETA: 0s - loss: 0.8377 - price_loss: 0.8377 - type_loss: 3.1847 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0017WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 137s 5s/step - loss: 0.8377 - price_loss: 0.8377 - type_loss: 3.1847 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0017 - val_loss: 0.8295 - val_price_loss: 0.8295 - val_type_loss: 3.1849 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 19/50
30/30 [==============================] - ETA: 0s - loss: 0.8376 - price_loss: 0.8376 - type_loss: 3.1849 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0017WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 137s 5s/step - loss: 0.8376 - price_loss: 0.8376 - type_loss: 3.1849 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0017 - val_loss: 0.8294 - val_price_loss: 0.8294 - val_type_loss: 3.1850 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
Epoch 20/50
30/30 [==============================] - ETA: 0s - loss: 0.8377 - price_loss: 0.8377 - type_loss: 3.1848 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0017WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 137s 5s/step - loss: 0.8377 - price_loss: 0.8377 - type_loss: 3.1848 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0017 - val_loss: 0.8294 - val_price_loss: 0.8294 - val_type_loss: 3.1850 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0014
plot_loss_accuracy_curves(history_model_conv2d_image_only)
________________________________________________________________________________________________________________________________________________________________________________________________________
submission_one_input(model_conv2d_image_only , "conv2d_image_only" ,x_test_image )
model_save(model_conv2d_image_only , "conv2d_image_only")
230/230 [==============================] - 28s 124ms/step probabilities of predicted price [[0.620392 0.3153832 0.06422475] [0.620392 0.3153832 0.06422475] [0.620392 0.3153832 0.06422475] ... [0.620392 0.3153832 0.06422475] [0.620392 0.3153832 0.06422475] [0.620392 0.3153832 0.06422475]] ____________________________________________________________________________________________________ values of of predicted price : [0 0 0 ... 0 0 0] counts of classes predicted in submitted file <bound method DataFrame.value_counts of id price 0 0 0 1 1 0 2 2 0 3 3 0 4 4 0 ... ... ... 7355 7626 0 7356 7627 0 7357 7628 0 7358 7629 0 7359 7630 0 [7360 rows x 2 columns]>
flatten_drop = image_model(conv2d=True , drop = True)
model_drop_image_only = build_model_one_input(flatten_drop,in_image, 1 , 0 , weights_focal = False)
history_model_drop_image_only, model_drop_image_only = fit_model_one_input(model_drop_image_only, x_train_image)
Model: "model_8"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_2 (InputLayer) [(None, 128, 128, 2 0 []
)]
conv2d_12 (Conv2D) (None, 124, 124, 64 3264 ['input_2[0][0]']
)
dropout_9 (Dropout) (None, 124, 124, 64 0 ['conv2d_12[0][0]']
)
max_pooling2d_12 (MaxPooling2D (None, 62, 62, 64) 0 ['dropout_9[0][0]']
)
conv2d_13 (Conv2D) (None, 56, 56, 64) 200768 ['max_pooling2d_12[0][0]']
dropout_10 (Dropout) (None, 56, 56, 64) 0 ['conv2d_13[0][0]']
max_pooling2d_13 (MaxPooling2D (None, 28, 28, 64) 0 ['dropout_10[0][0]']
)
conv2d_14 (Conv2D) (None, 13, 13, 32) 524320 ['max_pooling2d_13[0][0]']
dropout_11 (Dropout) (None, 13, 13, 32) 0 ['conv2d_14[0][0]']
max_pooling2d_14 (MaxPooling2D (None, 6, 6, 32) 0 ['dropout_11[0][0]']
)
flatten_4 (Flatten) (None, 1152) 0 ['max_pooling2d_14[0][0]']
price (Dense) (None, 3) 3459 ['flatten_4[0][0]']
type (Dense) (None, 24) 27672 ['flatten_4[0][0]']
==================================================================================================
Total params: 759,483
Trainable params: 759,483
Non-trainable params: 0
__________________________________________________________________________________________________
Epoch 1/50
30/30 [==============================] - ETA: 0s - loss: 94.2113 - price_loss: 94.2113 - type_loss: 154.2825 - price_sparse_categorical_accuracy: 0.5131 - type_sparse_categorical_accuracy: 0.2667WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 161s 5s/step - loss: 94.2113 - price_loss: 94.2113 - type_loss: 154.2825 - price_sparse_categorical_accuracy: 0.5131 - type_sparse_categorical_accuracy: 0.2667 - val_loss: 1.0762 - val_price_loss: 1.0762 - val_type_loss: 3.1657 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.7688
Epoch 2/50
30/30 [==============================] - ETA: 0s - loss: 1.0875 - price_loss: 1.0875 - type_loss: 3.1875 - price_sparse_categorical_accuracy: 0.6169 - type_sparse_categorical_accuracy: 0.4174WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 160s 5s/step - loss: 1.0875 - price_loss: 1.0875 - type_loss: 3.1875 - price_sparse_categorical_accuracy: 0.6169 - type_sparse_categorical_accuracy: 0.4174 - val_loss: 1.0402 - val_price_loss: 1.0402 - val_type_loss: 3.1385 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.7688
Epoch 3/50
30/30 [==============================] - ETA: 0s - loss: 0.9976 - price_loss: 0.9976 - type_loss: 3.0965 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.4143WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 160s 5s/step - loss: 0.9976 - price_loss: 0.9976 - type_loss: 3.0965 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.4143 - val_loss: 0.9997 - val_price_loss: 0.9997 - val_type_loss: 3.1143 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.7688
Epoch 4/50
30/30 [==============================] - ETA: 0s - loss: 0.9423 - price_loss: 0.9423 - type_loss: 3.0600 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.3119WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 159s 5s/step - loss: 0.9423 - price_loss: 0.9423 - type_loss: 3.0600 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.3119 - val_loss: 0.9518 - val_price_loss: 0.9518 - val_type_loss: 3.0915 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0225
Epoch 5/50
30/30 [==============================] - ETA: 0s - loss: 0.8870 - price_loss: 0.8870 - type_loss: 3.0271 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.2329WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 159s 5s/step - loss: 0.8870 - price_loss: 0.8870 - type_loss: 3.0271 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.2329 - val_loss: 0.9051 - val_price_loss: 0.9051 - val_type_loss: 3.0723 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0225
Epoch 6/50
30/30 [==============================] - ETA: 0s - loss: 0.8509 - price_loss: 0.8509 - type_loss: 3.0021 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.1568WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 160s 5s/step - loss: 0.8509 - price_loss: 0.8509 - type_loss: 3.0021 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.1568 - val_loss: 0.8738 - val_price_loss: 0.8738 - val_type_loss: 3.0600 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0225
Epoch 7/50
30/30 [==============================] - ETA: 0s - loss: 0.8398 - price_loss: 0.8398 - type_loss: 2.9893 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.1200WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 163s 5s/step - loss: 0.8398 - price_loss: 0.8398 - type_loss: 2.9893 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.1200 - val_loss: 0.8609 - val_price_loss: 0.8609 - val_type_loss: 3.0550 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0225
Epoch 8/50
30/30 [==============================] - ETA: 0s - loss: 0.8382 - price_loss: 0.8382 - type_loss: 2.9838 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0882WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 160s 5s/step - loss: 0.8382 - price_loss: 0.8382 - type_loss: 2.9838 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0882 - val_loss: 0.8559 - val_price_loss: 0.8559 - val_type_loss: 3.0533 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0225
Epoch 9/50
30/30 [==============================] - ETA: 0s - loss: 0.8378 - price_loss: 0.8378 - type_loss: 2.9843 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0831WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 161s 5s/step - loss: 0.8378 - price_loss: 0.8378 - type_loss: 2.9843 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0831 - val_loss: 0.8561 - val_price_loss: 0.8561 - val_type_loss: 3.0534 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0225
Epoch 10/50
30/30 [==============================] - ETA: 0s - loss: 0.8378 - price_loss: 0.8378 - type_loss: 2.9837 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0865WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 160s 5s/step - loss: 0.8378 - price_loss: 0.8378 - type_loss: 2.9837 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0865 - val_loss: 0.8554 - val_price_loss: 0.8554 - val_type_loss: 3.0532 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0225
Epoch 11/50
30/30 [==============================] - ETA: 0s - loss: 0.8374 - price_loss: 0.8374 - type_loss: 2.9840 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0817WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 161s 5s/step - loss: 0.8374 - price_loss: 0.8374 - type_loss: 2.9840 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0817 - val_loss: 0.8546 - val_price_loss: 0.8546 - val_type_loss: 3.0529 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0225
Epoch 12/50
30/30 [==============================] - ETA: 0s - loss: 0.8387 - price_loss: 0.8387 - type_loss: 2.9837 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0790WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 160s 5s/step - loss: 0.8387 - price_loss: 0.8387 - type_loss: 2.9837 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0790 - val_loss: 0.8544 - val_price_loss: 0.8544 - val_type_loss: 3.0529 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0225
Epoch 13/50
30/30 [==============================] - ETA: 0s - loss: 0.8373 - price_loss: 0.8373 - type_loss: 2.9842 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0805WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 160s 5s/step - loss: 0.8373 - price_loss: 0.8373 - type_loss: 2.9842 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0805 - val_loss: 0.8553 - val_price_loss: 0.8553 - val_type_loss: 3.0535 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0225
Epoch 14/50
30/30 [==============================] - ETA: 0s - loss: 0.8382 - price_loss: 0.8382 - type_loss: 2.9837 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0860WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 160s 5s/step - loss: 0.8382 - price_loss: 0.8382 - type_loss: 2.9837 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0860 - val_loss: 0.8549 - val_price_loss: 0.8549 - val_type_loss: 3.0536 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0225
Epoch 15/50
30/30 [==============================] - ETA: 0s - loss: 0.8379 - price_loss: 0.8379 - type_loss: 2.9852 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0812WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 160s 5s/step - loss: 0.8379 - price_loss: 0.8379 - type_loss: 2.9852 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0812 - val_loss: 0.8549 - val_price_loss: 0.8549 - val_type_loss: 3.0538 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0225
Epoch 16/50
30/30 [==============================] - ETA: 0s - loss: 0.8376 - price_loss: 0.8376 - type_loss: 2.9848 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0800WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 161s 5s/step - loss: 0.8376 - price_loss: 0.8376 - type_loss: 2.9848 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0800 - val_loss: 0.8545 - val_price_loss: 0.8545 - val_type_loss: 3.0534 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0225
Epoch 17/50
30/30 [==============================] - ETA: 0s - loss: 0.8380 - price_loss: 0.8380 - type_loss: 2.9844 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0858WARNING:tensorflow:Can save best model only with val_accuracy available, skipping.
30/30 [==============================] - 162s 5s/step - loss: 0.8380 - price_loss: 0.8380 - type_loss: 2.9844 - price_sparse_categorical_accuracy: 0.6196 - type_sparse_categorical_accuracy: 0.0858 - val_loss: 0.8556 - val_price_loss: 0.8556 - val_type_loss: 3.0541 - val_price_sparse_categorical_accuracy: 0.6241 - val_type_sparse_categorical_accuracy: 0.0225
plot_loss_accuracy_curves(history_model_drop_image_only)
________________________________________________________________________________________________________________________________________________________________________________________________________
submission_one_input(model_drop_image_only , "drop_image_only" ,x_test_image )
model_save(model_drop_image_only , "drop_image_only")
230/230 [==============================] - 28s 123ms/step probabilities of predicted price [[0.5381364 0.33909425 0.12276933] [0.5381364 0.33909425 0.12276933] [0.5381364 0.33909425 0.12276933] ... [0.5381364 0.33909425 0.12276933] [0.5381364 0.33909425 0.12276933] [0.5381364 0.33909425 0.12276933]] ____________________________________________________________________________________________________ values of of predicted price : [0 0 0 ... 0 0 0] counts of classes predicted in submitted file <bound method DataFrame.value_counts of id price 0 0 0 1 1 0 2 2 0 3 3 0 4 4 0 ... ... ... 7355 7626 0 7356 7627 0 7357 7628 0 7358 7629 0 7359 7630 0 [7360 rows x 2 columns]>
Observation
No,A fully-connected model is not a good option for sequential data like time-series or natural language processing
No, they have some limitations when working with raw image data, such as they don't preserve the spatial information of the image. To tackle these issues, convolutional neural networks (CNNs) have been developed precisely for image data, which can preserve spatial information and perform much better than fully-connected models.
Gradient vanishing and gradient explosion are two common problems that occur during the training of deep neural networks, particularly in recurrent neural networks (RNNs).
On the other hand, gradient explosion refers to the opposite problem, where the gradient signal becomes very large and causes the network weights to update too much, which can make the training unstable.
GRU (Gated Recurrent Unit) and LSTM (Long Short-Term Memory) are two types of RNNs that are designed to mitigate these problems by incorporating gating mechanisms. Gating mechanisms allow the network to selectively filter the information that it receives, which helps to regulate the flow of gradients during training.
In LSTM, the gating mechanism consists of three gates: the input gate, forget gate, and output gate. The input gate regulates the amount of new information that is introduced into the network, the forget gate determines what information is retained or forgotten from the previous state, and the output gate regulates the amount of information that is outputted to the next state.
Similarly, in GRU, the gating mechanism consists of two gates: the update gate and reset gate. The update gate controls how much of the new information is incorporated into the current state, while the reset gate controls how much of the previous state is forgotten. By using these gates, GRU and LSTM are able to selectively filter the gradients and prevent them from vanishing or exploding too much, which helps to improve the performance of the network.
Multi-objective/multi-task learning is a machine learning approach where a model is trained to solve multiple related tasks or objectives simultaneously. The idea behind multi-task learning is that different tasks can share common features and learning these common features can improve the model's performance on all tasks.
Multi-modality learning is another machine learning approach that involves learning from multiple types of data or modalities. The idea behind multi-modality learning is that incorporating information from different modalities can help improve the model's performance.The multimodal learning model is capable of supplying a missing modality based on observed ones. The multimodal learning model combines two deep Boltzmann machines, each corresponding to one modality. An additional hidden layer is placed on top of the two Boltzmann Machines to produce the joint representation.
In this assignment, we use both multi-task learning and multi-modality learning. We train a single model to perform two related tasks: price and type. Additionally, we incorporate information from both image and text modalities to train our model. The image and text modalities are fused at different levels of the model architecture to learn the joint representation of both modalities.These two learning mechanisms can be used seperately.
In this assignment, we will combine them together in some trials and we used them seperately other trials.
our model is defined as:
Inputs (two modalities):
Summary (text data): using only embedding layer /LSTM / GRU / Bi-Directional
Image (image data)
Outputs (two predictions):
price category [0,1,2]
Type (optional) we can predict it or not.
XGBoost, LightGBM, and CatBoost are all popular gradient boosting frameworks that are used for supervised learning tasks such as classification and regressionused. While they share many similarities, there are some distinct differences among the three:
Performance: LightGBM has been found to be faster than XGBoost, while CatBoost is designed to handle categorical data well.
Memory usage: LightGBM has a lower memory footprint compared to XGBoost and CatBoost because of its ability to perform better data partitioning.
Handling missing data: CatBoost can handle missing data more efficiently than XGBoost and LightGBM.
Regularization: XGBoost is considered to be better at handling regularization as it allows for various types of regularization such as L1, L2, and Elastic net.
Feature engineering: CatBoost has built-in feature engineering capabilities, which means it can automatically handle different types of features in the data, such as categorical, numerical, and text.
In summary, these frameworks have an extensive list of tunable hyperparameters that affect learning and eventual performance. All of them have the ability to execute on either CPUs or GPUs for accelerated learning. The main differences among XGBoost, LightGBM, and CatBoost lie in their performance, memory usage, handling of missing data, regularization, and feature engineering capabilities. The choice among these three models depends on the specific requirements and characteristics of the problem at hand.
LightGBM uses a novel technique of Gradient-based One-Side Sampling (GOSS) to filter out the data instances for finding a split value while
XGBoost uses pre-sorted algorithm &Histogram-based algorithm for computing the best split.
CatBoost tree can predict extra ordinarily faster than either XGBoost or LightGBM.
The important part before starting to do any thing, we understand the problem very well to be able to be soundness about features you need and features have effect on your prediction.
One of the biggest problems when people prepare to post a new listing on airbnb is, how much should one ask for? Of course the most intuitive way is to check how other similar postings price their apartment/house.
Our goal of this problem is to predict the listing price based on the listing characteristics 🔥🔥, in this way to optimize user experience and lower the bar to be a new host😍 !
This is a multimodality task which is to estimate the price of an Airbnb listing using multi class classification based on the photo and description provided.
The price of a new listing, as well as the type of rental being promoted, is predicted using a multi-objective method.
The data is presented in the form of text and images , there is an image and a text summary for each listing.
Type and price range are two of the model's outputs.
1- Importing the Libraries & Loading the Data
2- Data exploration : we used pandas and matplotlib libraries are used to plot the data and .
3- Data Preprocessing: In this step, we cleaned data from missing values ,duplicated rows ,high cardinality features and etc .Also, convert the categorical data to numirical data to make it ready for any type of model classification . For Text Preprocessing :
- Tokenization and remove regex and special characters
- converting text to integer IDs
- Remove any duplicates
- Translate text in aother language
For Image preprocessing:
- Resize the image to be (128X128 x 2)
and for the pretrained model vgg16 we resize it into (48x48x3)
4- Models:
- Model with Conv2d & Dropout layer
- Model with LSTM layer for text
- Model with GRU layer for text
- Model with Bi-directional Recurrent with LSTM layer
- Model with average embedding
- Multi-objective learning with Transfer Learning pretrained model
Small image dimensions (128X128) may provide problems because they may not include enough detail or information to effectively anticipate pricing ranges.Also, when I try to make it large , the training time is large and ram is crash.It needs powerful resourses.
There are different languages that we must deal with it, and there are some languages that we cannot translate. We will talk about them in detail in the preprocessing trial_2 .
On any Vacation rental platforms, In the agreement between the lessor and the lessee we may want to find answers to the following questions in near real-time:
Does a given piece of information contradict the other?
Does a given piece of information imply the other?
So, the impact of this solution could assist improve relationships and establish confidence with consumers renting these properties by streamlining the workflow for using Airbnb as a host.There is a high impact in optimizing user experience and lower the bar to be a new host
Airbnb pricing is important to get right, particularly in big cities like London where there is lots of competition and even small differences in prices can make a big difference. It is also a difficult thing to do correctly — price too high and no one will book. Price too low and you’ll be missing out on a lot of potential income.
The experimental protocol or procedure used is as follows: